{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/what-do-learning-dynamics-reveal-about","title":"What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?","arxiv_id":"2411.07681","date":"2024-11-12","proceeding":null,"authors":["Katie Kang","Amrith Setlur","Dibya Ghosh","Jacob Steinhardt","Claire Tomlin","Sergey Levine","Aviral Kumar"],"abstract":"Despite the remarkable capabilities of modern large language models (LLMs), the mechanisms behind their problem-solving abilities remain elusive. In this work, we aim to better understand how the learning dynamics of LLM finetuning shapes downstream generalization. Our analysis focuses on reasoning tasks, whose problem structure allows us to distinguish between memorization (the exact replication of reasoning steps from the training data) and performance (the correctness of the final solution). We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy: the accuracy of model samples on training queries before they begin to copy the exact reasoning steps from the training set. On the dataset level, this metric is able to reliably predict test accuracy, achieving $R^2$ of around or exceeding 0.9 across various models (Llama3 8, Gemma2 9B), datasets (GSM8k, MATH), and training configurations. On a per-example level, this metric is also indicative of whether individual model predictions are robust to perturbations in the training query. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies. We focus on data curation as an example, and show that prioritizing examples with low pre-memorization accuracy leads to 1.5-2x improvements in data efficiency compared to i.i.d. data scaling, and outperforms other standard data curation techniques.","url_abs":"https://arxiv.org/abs/2411.07681v2","url_pdf":"https://arxiv.org/pdf/2411.07681v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"what-do-learning-dynamics-reveal-about","repo_url":"https://github.com/katiekang1998/reasoning_generalization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"},{"task_slug":"memorization","task_name":"Memorization"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.07681","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.07681"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/katiekang1998/reasoning_generalization","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"20063c675d638a77","entry":"is_equiv","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"utils.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20063c675d638a77"}},{"code_sha256_prefix":"0b14c648516c38a7","entry":"last_boxed_only_string","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"math_eval_samples.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/math_eval_samples.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b14c648516c38a7"}},{"code_sha256_prefix":"f3bbe264b05aadd3","entry":"remove_boxed","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"math_eval_samples.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/math_eval_samples.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3bbe264b05aadd3"}},{"code_sha256_prefix":"ca893ca3034c4008","entry":"extract_latex","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"gsm8k_eval_samples.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/gsm8k_eval_samples.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca893ca3034c4008"}},{"code_sha256_prefix":"32ddc057ac01975c","entry":"get_aug_answer","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"gsm8k_eval_samples.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/gsm8k_eval_samples.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32ddc057ac01975c"}},{"code_sha256_prefix":"10f69d6b83d87ed5","entry":"preprocess","repo":"katiekang1998/reasoning_generalization","repo_kind":"official","path":"utils.py","file_url":"https://github.com/katiekang1998/reasoning_generalization/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10f69d6b83d87ed5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}