{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compositional-obverter-communication-learning","title":"Compositional Obverter Communication Learning From Raw Visual Input","arxiv_id":"1804.02341","date":"2018-04-06","proceeding":"ICLR 2018 1","authors":["Edward Choi","Angeliki Lazaridou","Nando de Freitas"],"abstract":"One of the distinguishing aspects of human language is its compositionality,\nwhich allows us to describe complex environments with limited vocabulary.\nPreviously, it has been shown that neural network agents can learn to\ncommunicate in a highly structured, possibly compositional language based on\ndisentangled input (e.g. hand- engineered features). Humans, however, do not\nlearn to communicate based on well-summarized features. In this work, we train\nneural agents to simultaneously develop visual perception from raw image\npixels, and learn to communicate with a sequence of discrete symbols. The\nagents play an image description game where the image contains factors such as\ncolors and shapes. We train the agents using the obverter technique where an\nagent introspects to generate messages that maximize its own understanding.\nThrough qualitative analysis, visualization and a zero-shot test, we show that\nthe agents can develop, out of raw image pixels, a language with compositional\nproperties, given a proper pressure from the environment.","url_abs":"http://arxiv.org/abs/1804.02341v1","url_pdf":"http://arxiv.org/pdf/1804.02341v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compositional-obverter-communication-learning","repo_url":"https://github.com/Joshua-Edinburgh/ms_thesis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"compositional-obverter-communication-learning","repo_url":"https://github.com/benbogin/obverter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"Image Description"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.02341","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1804.02341"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Joshua-Edinburgh/ms_thesis","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/benbogin/obverter","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bc5f93767e713176","entry":"decode","repo":"benbogin/obverter","repo_kind":"listed","path":"obverter.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/obverter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc5f93767e713176"}},{"code_sha256_prefix":"3d0597acef8618ab","entry":"get_relevant_state","repo":"benbogin/obverter","repo_kind":"listed","path":"model.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d0597acef8618ab"}},{"code_sha256_prefix":"ce77788ff7025990","entry":"get_sentence_lengths","repo":"benbogin/obverter","repo_kind":"listed","path":"model.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce77788ff7025990"}},{"code_sha256_prefix":"1fb071e4ad9ebe0e","entry":"load_images_dict","repo":"benbogin/obverter","repo_kind":"listed","path":"data.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1fb071e4ad9ebe0e"}},{"code_sha256_prefix":"dcae0559be9e8f17","entry":"pick_random_color","repo":"benbogin/obverter","repo_kind":"listed","path":"data.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcae0559be9e8f17"}},{"code_sha256_prefix":"4cd16e0890b91e7d","entry":"pick_random_object_type","repo":"benbogin/obverter","repo_kind":"listed","path":"data.py","file_url":"https://github.com/benbogin/obverter/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cd16e0890b91e7d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}