{"url":"/dataset/webtext","name":"WebText","full_name":null,"description_markdown":"**WebText** is an internal OpenAI corpus created by scraping web pages with emphasis on\r\ndocument quality. The authors scraped all outbound links from\r\nReddit which received at least 3\r\nkarma. The authors used the approach as a heuristic indicator for\r\nwhether other users found the link interesting, educational,\r\nor just funny.\r\n\r\nWebText contains the text subset of these 45 million links. It consists of over 8 million documents\r\nfor a total of 40 GB of text. All Wikipedia\r\ndocuments were removed from WebText since it is a common data source\r\nfor other datasets.","description_withheld":null,"homepage":"","introduced_date":"2019-02-14","introduced_date_note":null,"introduced_by":{"paper":"/paper/language-models-are-unsupervised-multitask","title":"Language Models are Unsupervised Multitask Learners","first_author":"Alec Radford","url":null},"license":{"name":"Private","url":null},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[],"languages":[{"name":"Kabyle","url":"/datasets/language/kabyle"}],"variants":["WebText"],"data_loaders":[],"num_papers_in_archive":425,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}