Datasets › C4
C4 (Colossal Clean Crawled Corpus)
C4 is a colossal, cleaned version of Common Crawl's web crawl corpus. It was based on Common Crawl dataset: https://commoncrawl.org. It was used to train the T5 text-to-text Transformer models.
The dataset can be downloaded in a pre-processed form from allennlp.
Benchmarks archive 2025-07-28
All 1 leaderboard whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
| First row (archive order) | Paper | Code | ||||
|---|---|---|---|---|---|---|
| Language Modelling | C4 | Primer Perplexity 12.35 | Primer: Searching for Efficient Transformers for... | labmlai/annotated_deep_learning_paper_implementations +3 | 9 | Compare |
Papers archive 2025-07-28
3 shown of 3 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 981. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.
| Date | Samples run Syntology | |||
|---|---|---|---|---|
| LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale | 4 | 4 | 15 Aug 2022 | ran 2 of 5 samples (3 unverified) |
| N-Grammer: Augmenting Transformers with latent n-grams | 2 | 2 | 13 Jul 2022 | ran 0 of 6 samples (6 unverified) |
| Primer: Searching for Efficient Transformers for Language Modeling | 4 | 3 | 17 Sep 2021 | ran 3 of 3 samples (0 unverified; 3 pointer-only for licence) |
Dataset loaders archive 2025-07-28
7 loaders as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
Unknown
Modalities archive 2025-07-28
Languages archive 2025-07-28
Variants archive 2025-07-28
- C4
- c4 en
2 variant names, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections