{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/linking-emergent-and-natural-languages-via-1","title":"Linking Emergent and Natural Languages via Corpus Transfer","arxiv_id":"2203.13344","date":"2022-03-24","proceeding":"ICLR 2022 4","authors":["Shunyu Yao","Mo Yu","Yang Zhang","Karthik R Narasimhan","Joshua B. Tenenbaum","Chuang Gan"],"abstract":"The study of language emergence aims to understand how human languages are shaped by perceptual grounding and communicative intent. Computational approaches to emergent communication (EC) predominantly consider referential games in limited domains and analyze the learned protocol within the game framework. As a result, it remains unclear how the emergent languages from these settings connect to natural languages or provide benefits in real-world language processing tasks, where statistical models trained on large text corpora dominate. In this work, we propose a novel way to establish such a link by corpus transfer, i.e. pretraining on a corpus of emergent language for downstream natural language tasks, which is in contrast to prior work that directly transfers speaker and listener parameters. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. For example, in a low-resource setup (modeling 2 million natural language tokens), pre-training on an emergent language corpus with just 2 million tokens reduces model perplexity by $24.6\\%$ on average across ten natural languages. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images. We find that our translation-based metric highly correlates with the downstream performance on modeling natural languages (for instance $\\rho=0.83$ on Hebrew), while topographic similarity, a popular metric in previous work, shows surprisingly low correlation ($\\rho=0.003$), hinting that simple properties like attribute disentanglement from synthetic domains might not capture the full complexities of natural language. Our findings also indicate potential benefits of moving language emergence forward with natural language resources and models.","url_abs":"https://arxiv.org/abs/2203.13344v1","url_pdf":"https://arxiv.org/pdf/2203.13344v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"linking-emergent-and-natural-languages-via-1","repo_url":"https://github.com/ysymyth/ec-nl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2203.13344","atlas_url":"https://app.syntology.ai/?focus=2203.13344","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.13344"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ysymyth/ec-nl","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ab4cee46c55a6804","entry":"DotAttn","repo":"ysymyth/ec-nl","repo_kind":"official","path":"ec-game/models.py","file_url":"https://github.com/ysymyth/ec-nl/blob/HEAD/ec-game/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab4cee46c55a6804"}},{"code_sha256_prefix":"4d08435ea0103a22","entry":"Speaker","repo":"ysymyth/ec-nl","repo_kind":"official","path":"ec-game/models.py","file_url":"https://github.com/ysymyth/ec-nl/blob/HEAD/ec-game/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d08435ea0103a22"}},{"code_sha256_prefix":"b656c9b77d891ba9","entry":"gumbel_softmax","repo":"ysymyth/ec-nl","repo_kind":"official","path":"ec-game/models.py","file_url":"https://github.com/ysymyth/ec-nl/blob/HEAD/ec-game/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b656c9b77d891ba9"}},{"code_sha256_prefix":"7a26c95f374badbf","entry":"gumbel_softmax_sample","repo":"ysymyth/ec-nl","repo_kind":"official","path":"ec-game/models.py","file_url":"https://github.com/ysymyth/ec-nl/blob/HEAD/ec-game/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a26c95f374badbf"}},{"code_sha256_prefix":"7ef50a6e2fd6e557","entry":"sample_gumbel","repo":"ysymyth/ec-nl","repo_kind":"official","path":"ec-game/models.py","file_url":"https://github.com/ysymyth/ec-nl/blob/HEAD/ec-game/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ef50a6e2fd6e557"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}