{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-few-shot-commonsense-knowledge","title":"Analyzing Commonsense Emergence in Few-shot Knowledge Models","arxiv_id":"2101.00297","date":"2021-01-01","proceeding":"AKBC 2021 10","authors":["Jeff Da","Ronan Le Bras","Ximing Lu","Yejin Choi","Antoine Bosselut"],"abstract":"Recently, commonsense knowledge models - pretrained language models (LM) fine-tuned on knowledge graph (KG) tuples - showed that considerable amounts of commonsense knowledge can be encoded in the parameters of large language models. However, as parallel studies show that LMs are poor hypothesizers of declarative commonsense relationships on their own, it remains unclear whether this knowledge is learned during pretraining or from fine-tuning on KG examples. To investigate this question, we train commonsense knowledge models in few-shot settings to study the emergence of their commonsense representation abilities. Our results show that commonsense knowledge models can rapidly adapt from limited examples, indicating that KG fine-tuning serves to learn an interface to encoded knowledge learned during pretraining. Importantly, our analysis of absolute, angular, and distributional parameter changes during few-shot fine-tuning provides novel insights into how this interface is learned.","url_abs":"https://arxiv.org/abs/2101.00297v3","url_pdf":"https://arxiv.org/pdf/2101.00297v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-few-shot-commonsense-knowledge","repo_url":"https://github.com/allenai/few-shot-comet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2101.00297","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2101.00297"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/few-shot-comet","reach":null}],"summary":{"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"1fc84c0fc44b92cf","entry":"get_s_l","repo":"allenai/few-shot-comet","repo_kind":"official","path":"section_5_interpretability/heatmaps/heatmaps.py","file_url":"https://github.com/allenai/few-shot-comet/blob/HEAD/section_5_interpretability/heatmaps/heatmaps.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fc84c0fc44b92cf"}},{"code_sha256_prefix":"75d1f06432c83e0f","entry":"load","repo":"allenai/few-shot-comet","repo_kind":"official","path":"section_5_interpretability/heatmaps/heatmaps.py","file_url":"https://github.com/allenai/few-shot-comet/blob/HEAD/section_5_interpretability/heatmaps/heatmaps.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75d1f06432c83e0f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}