Methods › Natural Language Processing › Language Models › OPT
OPT
Introduced by Susan Zhang et al. in OPT: Open Pre-trained Transformer Language Models
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
OPT is a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters. The model uses an AdamW optimizer and weight decay of 0.1. It follows a linear learning rate schedule, warming up from 0 to the maximum learning rate over the first 2000 steps in OPT-175B, or over 375M tokens in the smaller models, and decaying down to 10% of the maximum LR over 300B tokens. The batch sizes range from 0.5M to 4M depending on the model size and is kept constant throughout the course of training.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 11 in the graph, strongest link: official). This answer is about arXiv:2205.01068, the paper the archive credits with this method.
Origin paper of this answer, arXiv:2205.01068 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 14 ran (of which 2 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified (of 24 harvested samples) · 17 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 285 · with a code link: 123 · where Syntology ran a sample: 55 (39 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (55 of 285 tagged: 39 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument)
Page 1 of 3: papers 1 to 100 of 285, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Incentivizing High-quality Participation From Federated Learning Agents 20 Jun 2025 · 0 repositories · arXiv:2506.16731
-
Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs 16 Jun 2025 · 1 repository · arXiv:2506.13727
-
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization 16 Jun 2025 · 0 repositories · arXiv:2506.13541
-
TensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end Devices 16 Jun 2025 · 0 repositories · arXiv:2506.13514
-
FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed 10 Jun 2025 · 0 repositories · arXiv:2506.09034
-
Conservative Bias in Large Language Models: Measuring Relation Predictions 9 Jun 2025 · 0 repositories · arXiv:2506.08120
-
NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics 22 May 2025 · 0 repositories · arXiv:2505.16210
-
Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity 20 May 2025 · 1 repository · arXiv:2505.14884Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Optimizing Energy Consumption in Stochastic Production Systems: Using a Simulation-Based Approach for Stopping Policy 14 May 2025 · 0 repositories · arXiv:2505.11536
-
Anticipating Gaming to Incentivize Improvement: Guiding Agents in (Fair) Strategic Classification 8 May 2025 · 0 repositories · arXiv:2505.05594
-
SPAP: Structured Pruning via Alternating Optimization and Penalty Methods 6 May 2025 · 0 repositories · arXiv:2505.03373
-
Efficient Shapley Value-based Non-Uniform Pruning of Large Language Models 3 May 2025 · 0 repositories · arXiv:2505.01731
-
UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation 29 Apr 2025 · 1 repository · arXiv:2504.20500Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation 23 Apr 2025 · 0 repositories · arXiv:2504.16693
-
Transferable text data distillation by trajectory matching 14 Apr 2025 · 0 repositories · arXiv:2504.09818
-
Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs 24 Mar 2025 · 0 repositories · arXiv:2503.18377
-
MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processing 24 Mar 2025 · 0 repositories · arXiv:2503.18461
-
How does Bike Absence Influence Mode Shifts Among Dockless Bike-Sharing Users? Evidence From Nanjing, China 18 Mar 2025 · 0 repositories · arXiv:2503.14265
-
Large language models in finance : what is financial sentiment? 5 Mar 2025 · 0 repositories · arXiv:2503.03612
-
Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing 21 Feb 2025 · 1 repository · arXiv:2502.15618Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping 16 Feb 2025 · 1 repository · arXiv:2502.11104
-
The Odyssey of the Fittest: Can Agents Survive and Still Be Good? 8 Feb 2025 · 1 repository · arXiv:2502.05442
-
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation 19 Jan 2025 · 1 repository · arXiv:2501.11006
-
FASP: Fast and Accurate Structured Pruning of Large Language Models 16 Jan 2025 · 0 repositories · arXiv:2501.09412
-
Multilingual LLMs Struggle to Link Orthography and Semantics in Bilingual Word Processing 15 Jan 2025 · 1 repository · arXiv:2501.09127
-
Has an AI model been trained on your images? 11 Jan 2025 · 0 repositories · arXiv:2501.06399
-
Explore Activation Sparsity in Recurrent LLMs for Energy-Efficient Neuromorphic Computing 9 Jan 2025 · 1 repository · arXiv:2501.16337
-
From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Models 1 Jan 2025 · 0 repositories
-
Sentiment trading with large language models 26 Dec 2024 · 0 repositories · arXiv:2412.19245
-
RAGONITE: Iterative Retrieval on Induced Databases and Verbalized RDF for Conversational QA over KGs with RAG 23 Dec 2024 · 0 repositories · arXiv:2412.17690
-
Sharp Results for Hypothesis Testing with Risk-Sensitive Agents 21 Dec 2024 · 0 repositories · arXiv:2412.16452
-
Variability Need Not Imply Error: The Case of Adequate but Semantically Distinct Responses 20 Dec 2024 · 1 repository · arXiv:2412.15683
-
Sequence-Level Leakage Risk of Training Data in Large Language Models 15 Dec 2024 · 0 repositories · arXiv:2412.11302
-
RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting 10 Dec 2024 · 1 repository · arXiv:2412.07675
-
Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi 9 Dec 2024 · 1 repository · arXiv:2412.06333
-
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization 8 Dec 2024 · 0 repositories · arXiv:2412.06858
-
IterL2Norm: Fast Iterative L2-Normalization 6 Dec 2024 · 0 repositories · arXiv:2412.04778
-
The Impact of Generative AI on Student Churn and the Future of Formal Education 30 Nov 2024 · 0 repositories · arXiv:2412.00605
-
An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation 28 Nov 2024 · 0 repositories · arXiv:2411.19203
-
The Performance of the LSTM-based Code Generated by Large Language Models (LLMs) in Forecasting Time Series Data 27 Nov 2024 · 0 repositories · arXiv:2411.18731
-
Strategic Prompting for Conversational Tasks: A Comparative Analysis of Large Language Models Across Diverse Conversational Tasks 26 Nov 2024 · 0 repositories · arXiv:2411.17204
-
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format 24 Nov 2024 · 0 repositories · arXiv:2411.15982
-
Robust Detection of Watermarks for Large Language Models Under Human Edits 21 Nov 2024 · 1 repository · arXiv:2411.13868
-
Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models 14 Nov 2024 · 0 repositories · arXiv:2411.09837
-
Understanding the Effects of Human-written Paraphrases in LLM-generated Text Detection 6 Nov 2024 · 1 repository · arXiv:2411.03806
-
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM 3 Nov 2024 · 1 repository · arXiv:2411.01610
-
Reconstruction of Differentially Private Text Sanitization via Large Language Models 16 Oct 2024 · 0 repositories · arXiv:2410.12443
-
DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models 15 Oct 2024 · 1 repository · arXiv:2410.11988Syntology official (archive's flag): 4 ran · 7 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 3 pointer-only (licence)
-
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models 13 Oct 2024 · 0 repositories · arXiv:2410.09823
-
CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression 10 Oct 2024 · 0 repositories · arXiv:2410.07505
-
Generative Model for Less-Resourced Language with 1 billion parameters 9 Oct 2024 · 0 repositories · arXiv:2410.06898
-
Optimal Information Acquisition Strategies: The Case of Online Lending 7 Oct 2024 · 0 repositories · arXiv:2410.05539
-
Timer-XL: Long-Context Transformers for Unified Time Series Forecasting 7 Oct 2024 · 2 repositories · arXiv:2410.04803
-
Towards using Reinforcement Learning for Scaling and Data Replication in Cloud Systems 7 Oct 2024 · 0 repositories · arXiv:2410.11862
-
Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking 3 Oct 2024 · 0 repositories · arXiv:2410.14685
-
Harnessing Diversity for Important Data Selection in Pretraining Large Language Models 25 Sep 2024 · 0 repositories · arXiv:2409.16986
-
Analog Beamforming Aided by Full-Dimension One-Bit Chains 10 Sep 2024 · 0 repositories · arXiv:2409.06819
-
A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction 6 Sep 2024 · 1 repository · arXiv:2409.04150
-
Efficient LLM Context Distillation 3 Sep 2024 · 0 repositories · arXiv:2409.01930
-
Imagining from Images with an AI Storytelling Tool 21 Aug 2024 · 0 repositories · arXiv:2408.11517
-
LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models 20 Aug 2024 · 1 repository · arXiv:2408.10631
-
MoDeGPT: Modular Decomposition for Large Language Model Compression 19 Aug 2024 · 0 repositories · arXiv:2408.09632
-
WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models 18 Aug 2024 · 0 repositories · arXiv:2408.09459
-
The Nah Bandit: Modeling User Non-compliance in Recommendation Systems 15 Aug 2024 · 0 repositories · arXiv:2408.07897
-
Decentralized Health Intelligence Network (DHIN) 12 Aug 2024 · 0 repositories · arXiv:2408.06240
-
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression 10 Aug 2024 · 1 repository · arXiv:2408.05646Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (of 13 harvested samples) · 13 pointer-only (licence)
-
A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models 7 Aug 2024 · 0 repositories · arXiv:2408.03728
-
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons 6 Aug 2024 · 1 repository · arXiv:2408.03247
-
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs 3 Aug 2024 · 0 repositories · arXiv:2408.01803
-
A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder 30 Jul 2024 · 0 repositories · arXiv:2407.20485
-
Nudging Consent and the New Opt Out System to the Processing of Health Data in England 28 Jul 2024 · 0 repositories · arXiv:2407.19447
-
Impact of Model Size on Fine-tuned LLM Performance in Data-to-Text Generation: A State-of-the-Art Investigation 19 Jul 2024 · 0 repositories · arXiv:2407.14088
-
MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models 16 Jul 2024 · 0 repositories · arXiv:2407.11681
-
OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling 13 Jul 2024 · 1 repository · arXiv:2407.09887Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference 12 Jul 2024 · 0 repositories · arXiv:2407.09722
-
Randomness Helps Rigor: A Probabilistic Learning Rate Scheduler Bridging Theory and Deep Learning Practice 10 Jul 2024 · 0 repositories · arXiv:2407.07613
-
Sinkhorn algorithms and linear programming solvers for optimal partial transport problems 9 Jul 2024 · 0 repositories · arXiv:2407.06481
-
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations 4 Jul 2024 · 0 repositories · arXiv:2407.03604
-
FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models 24 Jun 2024 · 1 repository · arXiv:2406.16863Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 1 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified (of 15 harvested samples) · 4 pointer-only (licence)
-
ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models 24 Jun 2024 · 1 repository · arXiv:2406.16635Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Mitigating Social Biases in Language Models through Unlearning 19 Jun 2024 · 0 repositories · arXiv:2406.13551
-
Optimizing Large Model Training through Overlapped Activation Recomputation 13 Jun 2024 · 0 repositories · arXiv:2406.08756
-
CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark 5 Jun 2024 · 1 repository · arXiv:2406.03431
-
A Theory for Token-Level Harmonization in Retrieval-Augmented Generation 3 Jun 2024 · 0 repositories · arXiv:2406.00944
-
DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection 1 Jun 2024 · 1 repository · arXiv:2406.00345Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 9 harvested samples) · 4 pointer-only (licence)
-
Effective Interplay between Sparsity and Quantization: From Theory to Practice 31 May 2024 · 0 repositories · arXiv:2405.20935
-
Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs 31 May 2024 · 0 repositories · arXiv:2405.20835
-
Large Language Model Watermark Stealing With Mixed Integer Programming 30 May 2024 · 0 repositories · arXiv:2405.19677
-
Automatic Domain Adaptation by Transformers in In-Context Learning 27 May 2024 · 0 repositories · arXiv:2405.16819
-
Looking Backward: Streaming Video-to-Video Translation with Feature Banks 24 May 2024 · 1 repository · arXiv:2405.15757Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 12 harvested samples) · 12 pointer-only (licence)
-
Your Transformer is Secretly Linear 19 May 2024 · 1 repository · arXiv:2405.12250Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Scalarisation-based risk concepts for robust multi-objective optimisation 16 May 2024 · 1 repository · arXiv:2405.10221
-
Post Training Quantization of Large Language Models with Microscaling Formats 12 May 2024 · 0 repositories · arXiv:2405.07135
-
Robust and Explainable Fine-Grained Visual Classification with Transfer Learning: A Dual-Carriageway Framework 9 May 2024 · 0 repositories · arXiv:2405.05853
-
High Order Reasoning for Time Critical Recommendation in Evidence-based Medicine 5 May 2024 · 0 repositories · arXiv:2405.03010
-
Assessing Adversarial Robustness of Large Language Models: An Empirical Study 4 May 2024 · 0 repositories · arXiv:2405.02764
-
Parameter-Efficient Tuning Large Language Models for Graph Representation Learning 28 Apr 2024 · 0 repositories · arXiv:2404.18271
-
Learning to Beat ByteRL: Exploitability of Collectible Card Game Agents 25 Apr 2024 · 0 repositories · arXiv:2404.16689
-
Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach 22 Apr 2024 · 1 repository · arXiv:2404.14296
-
Towards Reliable Latent Knowledge Estimation in LLMs: Zero-Prompt Many-Shot Based Factual Knowledge Extraction 19 Apr 2024 · 1 repository · arXiv:2404.12957
Tasks archive 2025-07-28
20 shown of 247 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections