{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/language-conditioned-graph-networks-for","title":"Language-Conditioned Graph Networks for Relational Reasoning","arxiv_id":"1905.04405","date":"2019-05-10","proceeding":"ICCV 2019 10","authors":["Ronghang Hu","Anna Rohrbach","Trevor Darrell","Kate Saenko"],"abstract":"Solving grounded language tasks often requires reasoning about relationships between objects in the context of a given task. For example, to answer the question \"What color is the mug on the plate?\" we must check the color of the specific mug that satisfies the \"on\" relationship with respect to the plate. Recent work has proposed various methods capable of complex relational reasoning. However, most of their power is in the inference structure, while the scene is represented with simple local appearance features. In this paper, we take an alternate approach and build contextualized representations for objects in a visual scene to support relational reasoning. We propose a general framework of Language-Conditioned Graph Networks (LCGN), where each node represents an object, and is described by a context-aware representation from related objects through iterative message passing conditioned on the textual input. E.g., conditioning on the \"on\" relationship to the plate, the object \"mug\" gathers messages from the object \"plate\" to update its representation to \"mug on the plate\", which can be easily consumed by a simple classifier for answer prediction. We experimentally show that our LCGN approach effectively supports relational reasoning and improves performance across several tasks and datasets. Our code is available at http://ronghanghu.com/lcgn.","url_abs":"https://arxiv.org/abs/1905.04405v2","url_pdf":"https://arxiv.org/pdf/1905.04405v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"language-conditioned-graph-networks-for","repo_url":"https://github.com/ronghanghu/lcgn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"referring-expression-comprehension","task_name":"Referring Expression Comprehension"},{"task_slug":"relational-reasoning","task_name":"Relational Reasoning"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-clevr","task":"Visual Question Answering (VQA)","dataset":"CLEVR","model":"single-hop + LCGN (ours)","rank_in_archive_order":10,"of":15,"metrics":{"Accuracy":"97.9"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-gqa-test-dev","task":"Visual Question Answering (VQA)","dataset":"GQA test-dev","model":"single-hop + LCGN (ours)","rank_in_archive_order":8,"of":17,"metrics":{"Accuracy":"55.8"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-gqa-test-std","task":"Visual Question Answering (VQA)","dataset":"GQA test-std","model":"single-hop + LCGN (ours)","rank_in_archive_order":5,"of":7,"metrics":{"Accuracy":"56.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.04405","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.04405"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ronghanghu/lcgn","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"summary":{"unverified":10},"by_repo_kind":{"listed":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0157cc868f5a25e8","entry":"apply_mask2d","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/lcgn.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/lcgn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"0157cc868f5a25e8"}},{"code_sha256_prefix":"446bd7310789d977","entry":"attention_bbox_interpolation","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/vis.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/vis.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"446bd7310789d977"}},{"code_sha256_prefix":"6f35741d53c62ec6","entry":"attention_grid_interpolation","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/vis.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/vis.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6f35741d53c62ec6"}},{"code_sha256_prefix":"798f8641ab86db6a","entry":"bbox_regression","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/output_unit.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/output_unit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"798f8641ab86db6a"}},{"code_sha256_prefix":"148715c7eef6f9ca","entry":"classifier","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/output_unit.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/output_unit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"148715c7eef6f9ca"}},{"code_sha256_prefix":"1c9dd4910c33ae7c","entry":"embedding_op","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/input_unit.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/input_unit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1c9dd4910c33ae7c"}},{"code_sha256_prefix":"753f9303e5b182db","entry":"encoder","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/input_unit.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/input_unit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"753f9303e5b182db"}},{"code_sha256_prefix":"bf62d3b493e93fc6","entry":"hingeLoss","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/ops.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"bf62d3b493e93fc6"}},{"code_sha256_prefix":"3fdc0011dd4f6cda","entry":"inter2logits","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/ops.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"3fdc0011dd4f6cda"}},{"code_sha256_prefix":"2de8a558c3e97e3d","entry":"relu","repo":"ronghanghu/lcgn","repo_kind":"listed","path":"models_clevr/ops.py","file_url":"https://github.com/ronghanghu/lcgn/blob/HEAD/models_clevr/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2de8a558c3e97e3d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}