{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/are-emergent-abilities-in-large-language","title":"Are Emergent Abilities in Large Language Models just In-Context Learning?","arxiv_id":"2309.01809","date":"2023-09-04","proceeding":null,"authors":["Sheng Lu","Irina Bigoulaeva","Rachneet Sachdeva","Harish Tayyar Madabushi","Iryna Gurevych"],"abstract":"Large language models, comprising billions of parameters and pre-trained on extensive web-scale corpora, have been claimed to acquire certain capabilities without having been specifically trained on them. These capabilities, referred to as \"emergent abilities,\" have been a driving force in discussions regarding the potentials and risks of language models. A key challenge in evaluating emergent abilities is that they are confounded by model competencies that arise through alternative prompting techniques, including in-context learning, which is the ability of models to complete a task based on a few examples. We present a novel theory that explains emergent abilities, taking into account their potential confounding factors, and rigorously substantiate this theory through over 1000 experiments. Our findings suggest that purported emergent abilities are not truly emergent, but result from a combination of in-context learning, model memory, and linguistic knowledge. Our work is a foundational step in explaining language model performance, providing a template for their efficient use and clarifying the paradox of their ability to excel in some instances while faltering in others. Thus, we demonstrate that their capabilities should not be overestimated.","url_abs":"https://arxiv.org/abs/2309.01809v2","url_pdf":"https://arxiv.org/pdf/2309.01809v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"are-emergent-abilities-in-large-language","repo_url":"https://github.com/ukplab/on-emergence","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.01809","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.01809"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ukplab/on-emergence","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"2e0ff18c16875bd8","entry":"predict","repo":"ukplab/on-emergence","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ukplab/on-emergence/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2e0ff18c16875bd8"}},{"code_sha256_prefix":"7a2a70445337c927","entry":"prepare_data_bigbench","repo":"ukplab/on-emergence","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ukplab/on-emergence/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7a2a70445337c927"}},{"code_sha256_prefix":"db95ede9105abdd2","entry":"prepare_loader","repo":"ukplab/on-emergence","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ukplab/on-emergence/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"db95ede9105abdd2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}