Papers › Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

3 Apr 2023arXiv:2304.01373archive 2025-07-28

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal

How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce \textit{Pythia}, a suite of 16 LLMs all trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters. We provide public access to 154 checkpoints for each one of the 16 models, alongside tools to download and reconstruct their exact training dataloaders for further study. We intend \textit{Pythia} to facilitate research in many areas, and we present several case studies including novel results in memorization, term frequency effects on few-shot performance, and reducing gender bias. We demonstrate that this highly controlled setup can be used to yield novel insights toward LLMs and their training dynamics. Trained models, analysis code, training code, and training data can be found at \url{https://github.com/EleutherAI/pythia}.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

eleutherai/gpt-neox officialmentioned in papermentioned on GitHubpytorchApache-2.0 report
eleutherai/pythia officialmentioned in papermentioned on GitHubpytorchApache-2.0 report
Lightning-AI/lit-gpt mentioned on GitHubpytorch report
jzhang38/tinyllama mentioned on GitHubpytorchApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Common Sense ReasoningCoreference ResolutionLanguage ModellingMemorizationQuestion Answering

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Common Sense Reasoning ARC (Challenge) Pythia 12B (5-shot) Accuracy 36.8 #49 of 54 Archive leaderboard report
Common Sense Reasoning ARC (Challenge) Pythia 12B (0-shot) Accuracy 31.8 #51 of 54 Archive leaderboard report
Common Sense Reasoning ARC (Easy) Pythia 12B (5-shot) Accuracy 71.5 #28 of 47 Archive leaderboard report
Common Sense Reasoning ARC (Easy) Pythia 12B (0-shot) Accuracy 70.2 #33 of 47 Archive leaderboard report
Common Sense Reasoning WinoGrande Pythia 12B (5-shot) Accuracy 66.6 #43 of 77 Archive leaderboard report
Common Sense Reasoning WinoGrande Pythia 12B (0-shot) Accuracy 63.9 #47 of 77 Archive leaderboard report
Common Sense Reasoning WinoGrande Pythia 6.9B (0-shot) Accuracy 60.9 #49 of 77 Archive leaderboard report
Common Sense Reasoning WinoGrande Pythia 2.8B (0-shot) Accuracy 59.4 #52 of 77 Archive leaderboard report
Coreference Resolution Winograd Schema Challenge Pythia 12B (0-shot) Accuracy 54.8 #71 of 82 Archive leaderboard report
Coreference Resolution Winograd Schema Challenge Pythia 2.8B (0-shot) Accuracy 38.5 #79 of 82 Archive leaderboard report
Coreference Resolution Winograd Schema Challenge Pythia 6.9B (0-shot) Accuracy 36.5 #81 of 82 Archive leaderboard report
Coreference Resolution Winograd Schema Challenge Pythia 12B (5-shot) Accuracy 36.5 #82 of 82 Archive leaderboard report
Language Modelling LAMBADA Pythia 12B (0-shot) Accuracy 70.46 #22 of 37 Archive leaderboard report
Language Modelling LAMBADA Pythia 6.9B (0-shot) Accuracy 67.28 #26 of 37 Archive leaderboard report
Language Modelling LAMBADA Pythia 12B(Zero-Shot) Perplexity 3.92 #35 of 37 Archive leaderboard report
Language Modelling LAMBADA Pythia 6.9B(Zero-Shot) Perplexity 4.45 #36 of 37 Archive leaderboard report
Question Answering PIQA Pythia 12B (5-shot) Accuracy 76.7 #44 of 67 Archive leaderboard report
Question Answering PIQA Pythia 12B (0-shot) Accuracy 76 #46 of 67 Archive leaderboard report
Question Answering PIQA Pythia 6.9B (0-shot) Accuracy 75.2 #49 of 67 Archive leaderboard report
Question Answering PIQA Pythia 1B (5-shot) Accuracy 70.4 #57 of 67 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: Pythia

Pythia

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections