Browse State-of-the-Art › Memorization
Memorization
438 papers with code · 1 benchmark · 4 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| BIG-bench (Hindu Knowledge) (3 rows) | PaLM-540B (few-shot, k=5) | PaLM: Scaling Language Modeling with Pathways | code | Syntology 32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 1,088 · with a code link: 438 · where Syntology ran a sample: 177 (141 with a run with no instrument failure, 36 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (177 of 1,088 tagged: 141 with a run with no instrument failure, 36 where every run was a failure of Syntology's instrument)
Page 1 of 11: papers 1 to 100 of 1,088, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
25 Oct 2017 71 repositories listed Syntology community repositories only · 36 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 22 where Syntology's instrument failed) · 11 unverified (of 47 harvested samples) · 15 pointer-only (licence)We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
-
24 Jun 2016 39 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort.
-
31 Oct 2016 11 repositories listedThe ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence.
-
5 Apr 2022 7 repositories listed Syntology 32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (of 37 harvested samples)To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
-
9 Jun 2022 6 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
-
6 Jan 2022 6 repositories listed Syntology official (archive's flag): 8 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples)In this paper we propose to study generalization of neural networks on small algorithmically generated datasets.
-
1 Nov 2019 5 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Applying this augmentation to a strong Wikitext-103 LM, with neighbors drawn from the original training set, our kNN-LM achieves a new state-of-the-art perplexity of 15.
-
18 Apr 2018 5 repositories listed Syntology community repositories only · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)Deep learning with noisy labels is practically challenging, as the capacity of deep models is so high that they can totally memorize these noisy labels sooner or later during training.
-
Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models7 Jun 2023 4 repositories listed Syntology official (archive's flag): 2 ran · 53 ran (of which 0 constructed an object rather than computing a result; 30 with no instrument failure: 6 honoured, 1 violated, 23 with no contract checked; 23 where Syntology's instrument failed) · 10 unverified (of 63 harvested samples) · 12 pointer-only (licence)Comparing to 17 modern metrics for evaluating the overall performance, fidelity, diversity, rarity, and memorization of generative models, we find that the state-of-the-art perceptual realism of diffusion models as…
-
3 Apr 2023 4 repositories listedHow do large language models (LLMs) develop and evolve over the course of training?
-
22 Oct 2021 4 repositories listedThese observations require us to rethink the treatment of noisy labels, and we hope the availability of these two datasets would facilitate the development and evaluation of future learning with noisy label solutions.
-
1 Sep 2020 4 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning.
-
14 May 2019 4 repositories listed Syntology official: harvested, nothing ran · 15 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 1 honoured, 2 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (of 22 harvested samples) · 10 pointer-only (licence)In this article, we first frame the research problem of optimizing an adaptive and personalized spaced repetition scheduler when memorization concerns the application of underlying multiple skills.
-
22 May 2025 3 repositories listed Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)In this paper, we introduce R1-Searcher++, a novel framework designed to train LLMs to adaptively leverage both internal and external knowledge sources.
-
5 Feb 2025 3 repositories listedWhile conventional wisdom suggests that sophisticated reasoning tasks demand extensive training data (>100, 000 examples), we demonstrate that complex mathematical reasoning abilities can be effectively elicited with…
-
8 Jul 2024 3 repositories listedData owners may request the removal of their data from a trained model due to privacy or copyright concerns.
-
17 Jan 2022 3 repositories listedMachine Learning models face increased concerns regarding the storage of personal user data and adverse impacts of corrupted data like backdoors or systematic bias.
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
10 Jul 2021 3 repositories listedA dynamic transition mechanism is used to move from supervision loss in early learning to consistency loss for consensus of predictions among networks in the later stage.
-
6 Nov 2019 3 repositories listed Syntology 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)Sample selection approaches are popular in robust learning from noisy labels.
-
14 Jan 2019 3 repositories listedLearning with noisy labels is one of the hottest problems in weakly-supervised learning.
-
9 Feb 2016 3 repositories listedWe investigate a new method to augment recurrent neural networks with extra memory without increasing the number of network parameters.
-
14 Apr 2025 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)Scientific equation discovery is a fundamental task in the history of scientific progress, enabling the derivation of laws governing natural phenomena.
-
31 Dec 2024 2 repositories listedThis paper introduces a Hierarchical visual token Compression (HiCo) method designed for high-fidelity representation and a practical context modeling system VideoChat-Flash tailored for multimodal long-sequence…
-
30 Oct 2024 2 repositories listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 1 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples)Warm-starting neural network training by initializing networks with previously learned weights is appealing, as practical neural networks are often deployed under a continuous influx of new data.
-
17 Jun 2024 2 repositories listed Syntology official (archive's flag): 4 ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 3 pointer-only (licence)First, counterintuitively, we observe that pretraining on more data shows no significant improvement in the model's capability to acquire and maintain factual knowledge.
-
27 Dec 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)In a simple setting with direct supervision on the generative factors, we show how learning class-agnostic transformations offers a way to circumvent catastrophic forgetting and improve classification accuracy over time.
-
10 Dec 2023 2 repositories listedSince clean samples are easier distinguished by GMM with increasing noise, the memory bank can still maintain high quality at a high noise ratio.
-
10 Nov 2023 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)We propose the Data Contamination Quiz (DCQ), a simple and effective approach to detect data contamination in large language models (LLMs) and estimate the amount of it.
-
10 Nov 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 6 pointer-only (licence)However, this hypothesis heavily relies on the overfitting of target models, which will be mitigated by multiple regularization methods and the generalization of LLMs.
-
19 Oct 2023 2 repositories listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 2 honoured, 0 violated, 9 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 16 harvested samples) · 6 pointer-only (licence)
-
11 Oct 2023 2 repositories listed Syntology official (archive's flag): 11 ran · 13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
4 Oct 2023 2 repositories listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples) · 5 pointer-only (licence)
-
29 Sep 2023 2 repositories listed Syntology official (archive's flag): 3 ran · 11 ran (of which 1 constructed an object rather than computing a result; 7 with no instrument failure: 3 honoured, 3 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (of 16 harvested samples)
-
13 Jul 2023 2 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)
-
21 Apr 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
27 Feb 2023 2 repositories listed
-
18 Nov 2022 2 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
29 May 2022 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
14 Apr 2022 2 repositories listed
-
13 Jan 2022 2 repositories listed
-
6 Dec 2021 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 1 pointer-only (licence)
-
17 Nov 2021 2 repositories listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
26 Oct 2021 2 repositories listed Syntology official (archive's flag): 5 ran · 19 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 14 where Syntology's instrument failed) · 34 unverified (of 53 harvested samples)
-
7 Oct 2021 2 repositories listed Syntology official (archive's flag): 3 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 3 pointer-only (licence)
-
8 Sep 2021 2 repositories listed
-
27 Jul 2021 2 repositories listed
-
16 Jun 2021 2 repositories listed
-
19 Jan 2021 2 repositories listed
-
1 Dec 2020 2 repositories listed
-
17 Sep 2020 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
1 Jul 2020 2 repositories listed
-
30 Jun 2020 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
19 May 2020 2 repositories listed
-
1 Nov 2018 2 repositories listed
-
2 May 2018 2 repositories listed
-
17 Nov 2017 2 repositories listed
-
26 Oct 2017 2 repositories listed
-
16 Jun 2017 2 repositories listed
-
14 Jul 2025 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
17 Jun 2025 1 repository listed
-
17 Jun 2025 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
17 Jun 2025 1 repository listed
-
9 Jun 2025 1 repository listed
-
9 Jun 2025 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples) · 4 pointer-only (licence)
-
9 Jun 2025 1 repository listed
-
9 Jun 2025 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples)
-
4 Jun 2025 1 repository listed
-
29 May 2025 1 repository listed
-
29 May 2025 1 repository listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 19 harvested samples)
-
28 May 2025 1 repository listed
-
28 May 2025 1 repository listed
-
26 May 2025 1 repository listed
-
26 May 2025 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 5 harvested samples)
-
25 May 2025 1 repository listed
-
22 May 2025 1 repository listed
-
21 May 2025 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)
-
20 May 2025 1 repository listed
-
20 May 2025 1 repository listed
-
18 May 2025 1 repository listed
-
18 May 2025 1 repository listed
-
18 May 2025 1 repository listed
-
16 May 2025 1 repository listed
-
15 May 2025 1 repository listed
-
13 May 2025 1 repository listed
-
4 May 2025 1 repository listed
-
4 May 2025 1 repository listed
-
30 Apr 2025 1 repository listed
-
18 Apr 2025 1 repository listed
-
8 Apr 2025 1 repository listed
-
6 Apr 2025 1 repository listed
-
3 Apr 2025 1 repository listed
-
2 Apr 2025 1 repository listed
-
2 Apr 2025 1 repository listed
-
30 Mar 2025 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples)
-
24 Mar 2025 1 repository listed
-
24 Mar 2025 1 repository listed
-
17 Mar 2025 1 repository listed
-
9 Mar 2025 1 repository listed
Syntology lines on 41 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections