Methods › Natural Language Processing › Language Models › LLaMA
LLaMA
Introduced by Hugo Touvron et al. in LLaMA: Open and Efficient Foundation Language Models
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were subsequently proposed. The main difference with the original architecture are listed below.
- RMSNorm normalizing function is used to improve the training stability, by normalizing the input of each transformer sub-layer, instead of normalizing the output.
- The ReLU non-linearity is replaced by the SwiGLU activation function to improve performance.
- Absolute positional embeddings are removed and instead rotary positional embeddings (RoPE) are added at each layer of the network.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 56 in the graph, strongest link: official). This answer is about arXiv:2302.13971, the paper the archive credits with this method.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- LLaMA-Omni · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='LLaMA-Omni')
Origin paper of this answer, arXiv:2302.13971 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (of 58 harvested samples) · 4 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 1,062 · with a code link: 423 · where Syntology ran a sample: 143 (115 with a run with no instrument failure, 28 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (143 of 1,062 tagged: 115 with a run with no instrument failure, 28 where every run was a failure of Syntology's instrument)
Page 1 of 11: papers 1 to 100 of 1,062, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Making Language Model a Hierarchical Classifier and Generator 17 Jul 2025 · 1 repository · arXiv:2507.12930
-
Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions 16 Jul 2025 · 1 repository · arXiv:2507.11981
-
Seq vs Seq: An Open Suite of Paired Encoders and Decoders 15 Jul 2025 · 1 repository · arXiv:2507.11412Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 3 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)
-
Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores 10 Jul 2025 · 0 repositories · arXiv:2507.08143
-
Evaluation of Habitat Robotics using Large Language Models 8 Jul 2025 · 0 repositories · arXiv:2507.06157
-
MusiScene: Leveraging MU-LLaMA for Scene Imagination and Enhanced Video Background Music Generation 8 Jul 2025 · 0 repositories · arXiv:2507.05894
-
any4: Learned 4-bit Numeric Representation for LLMs 7 Jul 2025 · 1 repository · arXiv:2507.04610Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models 6 Jul 2025 · 0 repositories · arXiv:2507.04478
-
Large Language Models Acing Chartered Accountancy 26 Jun 2025 · 0 repositories · arXiv:2506.21031
-
CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation 25 Jun 2025 · 0 repositories · arXiv:2506.20128
-
OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling 25 Jun 2025 · 1 repository · arXiv:2506.20512Syntology official (archive's flag): 16 ran · 16 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 10 where Syntology's instrument failed) · 1 unverified (of 17 harvested samples) · 4 pointer-only (licence)
-
Can LLMs Replace Humans During Code Chunking? 24 Jun 2025 · 0 repositories · arXiv:2506.19897
-
Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives 22 Jun 2025 · 0 repositories · arXiv:2506.18116
-
Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Booster 22 Jun 2025 · 1 repository · arXiv:2506.18034
-
Shrinking the Generation-Verification Gap with Weak Verifiers 22 Jun 2025 · 0 repositories · arXiv:2506.18203
-
A Minimalist Optimizer Design for LLM Pretraining 20 Jun 2025 · 1 repository · arXiv:2506.16659Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
All is Not Lost: LLM Recovery without Checkpoints 18 Jun 2025 · 1 repository · arXiv:2506.15461
-
I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution 18 Jun 2025 · 0 repositories · arXiv:2506.17323
-
PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning 18 Jun 2025 · 0 repositories · arXiv:2506.15683
-
Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences 16 Jun 2025 · 2 repositories · arXiv:2506.13996
-
Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs 16 Jun 2025 · 1 repository · arXiv:2506.13727
-
Delving Into the Psychology of Machines: Exploring the Structure of Self-Regulated Learning via LLM-Generated Survey Responses 16 Jun 2025 · 0 repositories · arXiv:2506.13384
-
Evaluating Large Language Models for Phishing Detection, Self-Consistency, Faithfulness, and Explainability 16 Jun 2025 · 1 repository · arXiv:2506.13746
-
Language Models Enable Data-Augmented Synthesis Planning for Inorganic Materials 14 Jun 2025 · 0 repositories · arXiv:2506.12557
-
Training-free LLM Merging for Multi-task Learning 14 Jun 2025 · 1 repository · arXiv:2506.12379
-
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache 13 Jun 2025 · 0 repositories · arXiv:2506.11886
-
InfoFlood: Jailbreaking Large Language Models with Information Overload 13 Jun 2025 · 0 repositories · arXiv:2506.12274
-
LogiPlan: A Structured Benchmark for Logical Planning and Relational Reasoning in LLMs 12 Jun 2025 · 0 repositories · arXiv:2506.10527
-
Primender Sequence: A Novel Mathematical Construct for Testing Symbolic Inference and AI Reasoning 12 Jun 2025 · 0 repositories · arXiv:2506.10585
-
Contemporary AI foundation models increase biological weapons risk 12 Jun 2025 · 0 repositories · arXiv:2506.13798
-
Decomposing MLP Activations into Interpretable Features via Semi-Nonnegative Matrix Factorization 12 Jun 2025 · 1 repository · arXiv:2506.10920
-
CounselBench: A Large-Scale Expert Evaluation and Adversarial Benchmark of Large Language Models in Mental Health Counseling 10 Jun 2025 · 1 repository · arXiv:2506.08584
-
Fairness is Not Silence: Unmasking Vacuous Neutrality in Small Language Models 10 Jun 2025 · 0 repositories · arXiv:2506.08487
-
The impact of fine tuning in LLaMA on hallucinations for named entity extraction in legal documentation 10 Jun 2025 · 0 repositories · arXiv:2506.08827
-
Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments 8 Jun 2025 · 0 repositories · arXiv:2506.07232
-
Artificial Intelligence and Civil Discourse: How LLMs Moderate Climate Change Conversations 7 Jun 2025 · 0 repositories · arXiv:2506.12077
-
Benchmarking Large Language Models on Homework Assessment in Circuit Analysis 5 Jun 2025 · 0 repositories · arXiv:2506.06390
-
Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents 5 Jun 2025 · 1 repository · arXiv:2506.04649Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
Multiple-Choice Question Generation Using Large Language Models: Methodology and Educator Insights 5 Jun 2025 · 0 repositories · arXiv:2506.04851
-
Natural Language Interaction with Databases on Edge Devices in the Internet of Battlefield Things 5 Jun 2025 · 0 repositories · arXiv:2506.06396
-
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text 5 Jun 2025 · 0 repositories · arXiv:2506.05209
-
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability 4 Jun 2025 · 0 repositories · arXiv:2506.03655
-
Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate 4 Jun 2025 · 1 repository · arXiv:2506.04043
-
BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage 3 Jun 2025 · 1 repository · arXiv:2506.02479
-
Rethinking the effects of data contamination in Code Intelligence 3 Jun 2025 · 0 repositories · arXiv:2506.02791
-
Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem 3 Jun 2025 · 0 repositories · arXiv:2506.03295
-
WeightLoRA: Keep Only Necessary Adapters 3 Jun 2025 · 0 repositories · arXiv:2506.02724
-
Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification 2 Jun 2025 · 0 repositories · arXiv:2506.01631
-
Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection 1 Jun 2025 · 0 repositories · arXiv:2506.00955
-
DefenderBench: A Toolkit for Evaluating Language Agents in Cybersecurity Environments 31 May 2025 · 1 repository · arXiv:2506.00739
-
Machine vs Machine: Using AI to Tackle Generative AI Threats in Assessment 31 May 2025 · 0 repositories · arXiv:2506.02046
-
Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data 31 May 2025 · 1 repository · arXiv:2506.00469
-
MIRROR: Cognitive Inner Monologue Between Conversational Turns for Persistent Reflection and Reasoning in Conversational LLMs 31 May 2025 · 1 repository · arXiv:2506.00430
-
HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts 30 May 2025 · 1 repository · arXiv:2505.24722Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples)
-
Large Language Models are Locally Linear Mappings 30 May 2025 · 1 repository · arXiv:2505.24293
-
The Hallucination Dilemma: Factuality-Aware Reinforcement Learning for Large Reasoning Models 30 May 2025 · 1 repository · arXiv:2505.24630Syntology official (archive's flag): 5 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 1 pointer-only (licence)
-
TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis 30 May 2025 · 1 repository · arXiv:2505.24672
-
D-AR: Diffusion via Autoregressive Models 29 May 2025 · 1 repository · arXiv:2505.23660Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 0 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 7 pointer-only (licence)
-
Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach 29 May 2025 · 0 repositories · arXiv:2505.23953
-
Learning to Regulate: A New Event-Level Dataset of Capital Control Measures 29 May 2025 · 0 repositories · arXiv:2505.23025
-
Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution 29 May 2025 · 1 repository · arXiv:2505.24037
-
LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning 29 May 2025 · 0 repositories · arXiv:2506.03178
-
LUMION: Fast Fault Recovery for ML Jobs Using Programmable Optical Fabrics 29 May 2025 · 0 repositories · arXiv:2505.23105
-
Synthetic Document Question Answering in Hungarian 29 May 2025 · 1 repository · arXiv:2505.23008
-
Flexible Tool Selection through Low-dimensional Attribute Alignment of Vision and Language 28 May 2025 · 0 repositories · arXiv:2505.22146
-
SlimLLM: Accurate Structured Pruning for Large Language Models 28 May 2025 · 0 repositories · arXiv:2505.22689
-
AI-Supported Platform for System Monitoring and Decision-Making in Nuclear Waste Management with Large Language Models 27 May 2025 · 0 repositories · arXiv:2505.21741
-
Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspective 27 May 2025 · 0 repositories · arXiv:2505.20707
-
CaseEdit: Enhancing Localized Commonsense Reasoning via Null-Space Constrained Knowledge Editing in Small Parameter Language Models 26 May 2025 · 0 repositories · arXiv:2505.19383
-
DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph 26 May 2025 · 0 repositories · arXiv:2505.19956
-
Improving Multilingual Math Reasoning for African Languages 26 May 2025 · 0 repositories · arXiv:2505.19848
-
Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar 26 May 2025 · 0 repositories · arXiv:2505.19599
-
AI4Math: A Native Spanish Benchmark for University-Level Mathematical Reasoning in Large Language Models 25 May 2025 · 0 repositories · arXiv:2505.18978
-
Efficient Data Selection at Scale via Influence Distillation 25 May 2025 · 0 repositories · arXiv:2505.19051
-
Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk? 25 May 2025 · 0 repositories · arXiv:2505.18953
-
Efficient and Workload-Aware LLM Serving via Runtime Layer Swapping and KV Cache Resizing 24 May 2025 · 0 repositories · arXiv:2506.02006
-
How do managers' non-responses during earnings calls affect analyst forecasts 23 May 2025 · 0 repositories · arXiv:2505.18419
-
NeUQI: Near-Optimal Uniform Quantization Parameter Initialization 23 May 2025 · 1 repository · arXiv:2505.17595Syntology official (archive's flag): 6 ran · 6 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 14 unverified (of 20 harvested samples) · 20 pointer-only (licence)
-
LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing 22 May 2025 · 0 repositories · arXiv:2505.16491
-
When Safety Detectors Aren't Enough: A Stealthy and Effective Jailbreak Attack on LLMs via Steganographic Techniques 22 May 2025 · 0 repositories · arXiv:2505.16765
-
AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals 21 May 2025 · 0 repositories · arXiv:2505.15365
-
Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation 21 May 2025 · 0 repositories · arXiv:2505.16065
-
ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding 21 May 2025 · 1 repository · arXiv:2505.15046
-
Learning to Rank Chain-of-Thought: An Energy-Based Approach with Outcome Supervision 21 May 2025 · 0 repositories · arXiv:2505.14999
-
Multilingual Prompting for Improving LLM Generation Diversity 21 May 2025 · 0 repositories · arXiv:2505.15229
-
Small Language Models in the Real World: Insights from Industrial Text Classification 21 May 2025 · 0 repositories · arXiv:2505.16078
-
Choosing a Model, Shaping a Future: Comparing LLM Perspectives on Sustainability and its Relationship with AI 20 May 2025 · 0 repositories · arXiv:2505.14435
-
Do Language Models Use Their Depth Efficiently? 20 May 2025 · 1 repository · arXiv:2505.13898Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples)
-
Temporal Alignment of Time Sensitive Facts with Activation Engineering 20 May 2025 · 0 repositories · arXiv:2505.14158
-
A3 : an Analytical Low-Rank Approximation Framework for Attention 19 May 2025 · 0 repositories · arXiv:2505.12942
-
Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformers 19 May 2025 · 0 repositories · arXiv:2505.13737
-
Krikri: Advancing Open Large Language Models for Greek 19 May 2025 · 0 repositories · arXiv:2505.13772
-
Shadow-FT: Tuning Instruct via Base 19 May 2025 · 1 repository · arXiv:2505.12716
-
Extracting memorized pieces of (copyrighted) books from open-weight language models 18 May 2025 · 0 repositories · arXiv:2505.12546
-
KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache 18 May 2025 · 0 repositories · arXiv:2506.08018
-
Improving Fairness in LLMs Through Testing-Time Adversaries 17 May 2025 · 0 repositories · arXiv:2505.12100
-
Llama-Affinity: A Predictive Antibody Antigen Binding Model Integrating Antibody Sequences with Llama3 Backbone Architecture 17 May 2025 · 0 repositories · arXiv:2506.09052
-
MergeBench: A Benchmark for Merging Domain-Specialized LLMs 16 May 2025 · 1 repository · arXiv:2505.10833
-
Can Global XAI Methods Reveal Injected Bias in LLMs? SHAP vs Rule Extraction vs RuleSHAP 16 May 2025 · 1 repository · arXiv:2505.11189
-
PeerGuard: Defending Multi-Agent Systems Against Backdoor Attacks Through Mutual Reasoning 16 May 2025 · 1 repository · arXiv:2505.11642
Tasks archive 2025-07-28
20 shown of 450 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 158 |
| Language Modeling | 135 |
| Large Language Model | 115 |
| Question Answering | 66 |
| Quantization | 62 |
| GPU | 48 |
| Retrieval | 46 |
| RAG | 45 |
| Retrieval-augmented Generation | 39 |
| Text Generation | 39 |
| Code Generation | 36 |
| Benchmarking | 34 |
| Math | 34 |
| Prompt Engineering | 33 |
| In-Context Learning | 32 |
| Decision Making | 28 |
| parameter-efficient fine-tuning | 28 |
| Instruction Following | 27 |
| MMLU | 25 |
| Decoder | 24 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections