Papers › Language Models are Unsupervised Multitask Learners

Language Models are Unsupervised Multitask Learners

14 Feb 2019Preprint 2019 2archive 2025-07-28

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever

Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets. We demonstrate that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText. When conditioned on a document plus questions, the answers generated by the language model reach 55 F1 on the CoQA dataset - matching or exceeding the performance of 3 out of 4 baseline systems without using the 127,000+ training examples. The capacity of the language model is essential to the success of zero-shot task transfer and increasing it improves performance in a log-linear fashion across tasks. Our largest model, GPT-2, is a 1.5B parameter Transformer that achieves state of the art results on 7 out of 8 tested language modeling datasets in a zero-shot setting but still underfits WebText. Samples from the model reflect these improvements and contain coherent paragraphs of text. These findings suggest a promising path towards building language processing systems which learn to perform tasks from their naturally occurring demonstrations.

PaperPDFCode

Code

21 repositories listed; official and paper-mentioned ones first.

openai/gpt-2 officialmentioned in papertfNOASSERTION report
MS-P3/code5 mindspore report
lvyufeng/bert4ms mindsporeApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Common Sense ReasoningCoreference ResolutionData-to-Text GenerationDialogue State TrackingDocument SummarizationLanguage ModelingLanguage ModellingMachine TranslationMulti-Task LearningMulti-task Language UnderstandingQuestion AnsweringReading ComprehensionResponse GenerationSentiment AnalysisText GenerationTranslation

Datasets

Introduced by this paper, per the archive.

WebText

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Coreference Resolution Winograd Schema Challenge GPT-2-XL 1.5B Accuracy 70.7 #33 of 82 Archive leaderboard report
Dialogue State Tracking SIMMC2.0 GPT-2 Act F1 94.5 #4 of 5 Archive leaderboard report
Dialogue State Tracking SIMMC2.0 GPT-2 Slot F1 81.7 #4 of 5 Archive leaderboard report
Document Summarization CNN / Daily Mail GPT-2 ROUGE-1 29.34 #26 of 26 Archive leaderboard report
Document Summarization CNN / Daily Mail GPT-2 ROUGE-2 8.27 #26 of 26 Archive leaderboard report
Document Summarization CNN / Daily Mail GPT-2 ROUGE-L 26.58 #26 of 26 Archive leaderboard report
Language Modelling LAMBADA GPT-2 1.5B (Zero Shot) Accuracy 63.24 #29 of 37 Archive leaderboard report
Language Modelling LAMBADA GPT-2 1.5B (Zero Shot) Perplexity 8.63 #29 of 37 Archive leaderboard report
Language Modelling One Billion Word GPT-2 Number of params 1.54B #23 of 27 Archive leaderboard report
Language Modelling One Billion Word GPT-2 PPL 42.16 #23 of 27 Archive leaderboard report
Language Modelling Penn Treebank (Word Level) GPT-2 Params 1542M #3 of 43 Archive leaderboard report
Language Modelling Penn Treebank (Word Level) GPT-2 Test perplexity 35.76 #3 of 43 Archive leaderboard report
Language Modelling Text8 GPT-2 Bit per Character (BPC) 0.98 #1 of 24 Archive leaderboard report
Language Modelling Text8 GPT-2 Number of params 1542M #1 of 24 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Full Number of params 1542M #25 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Full Test perplexity 17.48 #25 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Large Number of params 774M #46 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Large Test perplexity 22.05 #46 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Medium Number of params 355M #63 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Medium Test perplexity 26.37 #63 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Small Number of params 124M #79 of 89 Archive leaderboard report
Language Modelling WikiText-103 GPT-2 Small Test perplexity 37.50 #79 of 89 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 Number of params 1542M #6 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 Test perplexity 18.34 #6 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (large) Number of params 762M #7 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (large) Test perplexity 19.93 #7 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (medium) Number of params 345M #8 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (medium) Test perplexity 22.76 #8 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (small) Number of params 117M #9 of 38 Archive leaderboard report
Language Modelling WikiText-2 GPT-2 (small) Test perplexity 29.41 #9 of 38 Archive leaderboard report
Language Modelling enwik8 GPT-2 (48 layers, h=1600) Bit per Character (BPC) 0.93 #1 of 42 Archive leaderboard report
Language Modelling enwik8 GPT-2 (48 layers, h=1600) Number of params 1542M #1 of 42 Archive leaderboard report
Question Answering FEVER Zero-shot EM 50 #7 of 8 Archive leaderboard report
Question Answering WebQuestions Zero-shot EM 43 #12 of 37 Archive leaderboard report
Response Generation SIMMC2.0 GPT-2 BLEU 19.2 #5 of 5 Archive leaderboard report
Sentiment Analysis IMDb GPT-2 Finetuned Accuracy 92.36 #31 of 49 Archive leaderboard report
Text Generation OpenWebText GPT2-124M eval_loss 3.12 #3 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: GPT-2

AdamAttentionAttention DropoutBPECosine AnnealingDense ConnectionsDiscriminative Fine-TuningDropoutGPT-2Layer NormalizationLinear LayerLinear Warmup With Cosine AnnealingMulti-Head AttentionResidual ConnectionSoftmaxWeight Decay

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections