{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/meaningful-learning-advancing-abstract","title":"Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance","arxiv_id":"2403.09085","date":"2024-03-14","proceeding":null,"authors":["Kai Xiong","Xiao Ding","Ting Liu","Bing Qin","Dongliang Xu","Qing Yang","Hongtao Liu","Yixin Cao"],"abstract":"Large language models (LLMs) have developed impressive performance and strong explainability across various reasoning scenarios, marking a significant stride towards mimicking human-like intelligence. Despite this, when tasked with several simple questions supported by a generic fact, LLMs often struggle to abstract and apply the generic fact to provide consistent and precise answers, revealing a deficiency in abstract reasoning abilities. This has sparked a vigorous debate about whether LLMs are genuinely reasoning or merely memorizing. In light of this, we design a preliminary study to quantify and delve into the abstract reasoning abilities of existing LLMs. Our findings reveal a substantial discrepancy between their general reasoning and abstract reasoning performances. To relieve this problem, we tailor an abstract reasoning dataset (AbsR) together with a meaningful learning paradigm to teach LLMs how to leverage generic facts for reasoning purposes. The results show that our approach not only boosts the general reasoning performance of LLMs but also makes considerable strides towards their capacity for abstract reasoning, moving beyond simple memorization or imitation to a more nuanced understanding and application of generic facts. The code is available at https://github.com/Waste-Wood/MeanLearn.","url_abs":"https://arxiv.org/abs/2403.09085v2","url_pdf":"https://arxiv.org/pdf/2403.09085v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"meaningful-learning-advancing-abstract","repo_url":"https://github.com/waste-wood/meanlearn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.09085","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.09085"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Waste-Wood/MeanLearn","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/waste-wood/meanlearn","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_fixture":1,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"622424aceb92bd33","entry":"bbh_freeform_postprocess","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"performance_bbh.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/performance_bbh.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"622424aceb92bd33"}},{"code_sha256_prefix":"e98d6e308bad1d37","entry":"clustering","repo":"waste-wood/meanlearn","repo_kind":"official","path":"clustering/clustering.py","file_url":"https://github.com/waste-wood/meanlearn/blob/HEAD/clustering/clustering.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e98d6e308bad1d37"}},{"code_sha256_prefix":"d208739629eb3b79","entry":"collate_fn","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"tools.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d208739629eb3b79"}},{"code_sha256_prefix":"a0379db9364cbc02","entry":"find_proper_index","repo":"waste-wood/meanlearn","repo_kind":"official","path":"clustering/clustering.py","file_url":"https://github.com/waste-wood/meanlearn/blob/HEAD/clustering/clustering.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0379db9364cbc02"}},{"code_sha256_prefix":"8461d5c9186434c7","entry":"process_input","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"reason_ppl.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/reason_ppl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8461d5c9186434c7"}},{"code_sha256_prefix":"3e8a91ae4fbf4a1a","entry":"process_instance","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"clustering/clustering.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/clustering/clustering.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e8a91ae4fbf4a1a"}},{"code_sha256_prefix":"e48818bea6862619","entry":"compute_loss","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"reason_ppl.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/reason_ppl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e48818bea6862619"}},{"code_sha256_prefix":"64708275fb08c548","entry":"process_classification","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"performance_bbh.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/performance_bbh.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64708275fb08c548"}},{"code_sha256_prefix":"95d8d0c584c301c2","entry":"process_generation","repo":"Waste-Wood/MeanLearn","repo_kind":"official","path":"performance_bbh.py","file_url":"https://github.com/Waste-Wood/MeanLearn/blob/HEAD/performance_bbh.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95d8d0c584c301c2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}