Methods › Natural Language Processing › Language Models › LLaMA
LLaMA
Introduced by Hugo Touvron et al. in LLaMA: Open and Efficient Foundation Language Models
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were subsequently proposed. The main difference with the original architecture are listed below.
- RMSNorm normalizing function is used to improve the training stability, by normalizing the input of each transformer sub-layer, instead of normalizing the output.
- The ReLU non-linearity is replaced by the SwiGLU activation function to improve performance.
- Absolute positional embeddings are removed and instead rotary positional embeddings (RoPE) are added at each layer of the network.
Papers archive 2025-07-28
30 shown of 1,062, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Making Language Model a Hierarchical Classifier and Generator 17 Jul 2025 · 1 repository · arXiv:2507.12930
-
Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions 16 Jul 2025 · 1 repository · arXiv:2507.11981
-
Seq vs Seq: An Open Suite of Paired Encoders and Decoders 15 Jul 2025 · 1 repository · arXiv:2507.11412Syntology ran 0 of 10 samples · 10 unverified
-
Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores 10 Jul 2025 · 0 repositories · arXiv:2507.08143
-
Evaluation of Habitat Robotics using Large Language Models 8 Jul 2025 · 0 repositories · arXiv:2507.06157
-
MusiScene: Leveraging MU-LLaMA for Scene Imagination and Enhanced Video Background Music Generation 8 Jul 2025 · 0 repositories · arXiv:2507.05894
-
any4: Learned 4-bit Numeric Representation for LLMs 7 Jul 2025 · 1 repository · arXiv:2507.04610Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
-
Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models 6 Jul 2025 · 0 repositories · arXiv:2507.04478
-
Large Language Models Acing Chartered Accountancy 26 Jun 2025 · 0 repositories · arXiv:2506.21031
-
CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation 25 Jun 2025 · 0 repositories · arXiv:2506.20128
-
OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling 25 Jun 2025 · 1 repository · arXiv:2506.20512Syntology ran 3 of 17 samples · 14 unverified
-
Can LLMs Replace Humans During Code Chunking? 24 Jun 2025 · 0 repositories · arXiv:2506.19897
-
Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives 22 Jun 2025 · 0 repositories · arXiv:2506.18116
-
Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Booster 22 Jun 2025 · 1 repository · arXiv:2506.18034
-
Shrinking the Generation-Verification Gap with Weak Verifiers 22 Jun 2025 · 0 repositories · arXiv:2506.18203
-
A Minimalist Optimizer Design for LLM Pretraining 20 Jun 2025 · 1 repository · arXiv:2506.16659Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
-
All is Not Lost: LLM Recovery without Checkpoints 18 Jun 2025 · 1 repository · arXiv:2506.15461
-
I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution 18 Jun 2025 · 0 repositories · arXiv:2506.17323
-
PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning 18 Jun 2025 · 0 repositories · arXiv:2506.15683
-
Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences 16 Jun 2025 · 2 repositories · arXiv:2506.13996
-
Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs 16 Jun 2025 · 1 repository · arXiv:2506.13727
-
Delving Into the Psychology of Machines: Exploring the Structure of Self-Regulated Learning via LLM-Generated Survey Responses 16 Jun 2025 · 0 repositories · arXiv:2506.13384
-
Evaluating Large Language Models for Phishing Detection, Self-Consistency, Faithfulness, and Explainability 16 Jun 2025 · 1 repository · arXiv:2506.13746
-
Language Models Enable Data-Augmented Synthesis Planning for Inorganic Materials 14 Jun 2025 · 0 repositories · arXiv:2506.12557
-
Training-free LLM Merging for Multi-task Learning 14 Jun 2025 · 1 repository · arXiv:2506.12379
-
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache 13 Jun 2025 · 0 repositories · arXiv:2506.11886
-
InfoFlood: Jailbreaking Large Language Models with Information Overload 13 Jun 2025 · 0 repositories · arXiv:2506.12274
-
LogiPlan: A Structured Benchmark for Logical Planning and Relational Reasoning in LLMs 12 Jun 2025 · 0 repositories · arXiv:2506.10527
-
Primender Sequence: A Novel Mathematical Construct for Testing Symbolic Inference and AI Reasoning 12 Jun 2025 · 0 repositories · arXiv:2506.10585
-
Contemporary AI foundation models increase biological weapons risk 12 Jun 2025 · 0 repositories · arXiv:2506.13798
Tasks archive 2025-07-28
20 shown of 450 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 158 |
| Language Modeling | 135 |
| Large Language Model | 115 |
| Question Answering | 66 |
| Quantization | 62 |
| GPU | 48 |
| Retrieval | 46 |
| RAG | 45 |
| Retrieval-augmented Generation | 39 |
| Text Generation | 39 |
| Code Generation | 36 |
| Benchmarking | 34 |
| Math | 34 |
| Prompt Engineering | 33 |
| In-Context Learning | 32 |
| Decision Making | 28 |
| parameter-efficient fine-tuning | 28 |
| Instruction Following | 27 |
| MMLU | 25 |
| Decoder | 24 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections