{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/explainable-and-explicit-visual-reasoning","title":"Explainable and Explicit Visual Reasoning over Scene Graphs","arxiv_id":"1812.01855","date":"2018-12-05","proceeding":"CVPR 2019 6","authors":["Jiaxin Shi","Hanwang Zhang","Juanzi Li"],"abstract":"We aim to dismantle the prevalent black-box neural architectures used in\ncomplex visual reasoning tasks, into the proposed eXplainable and eXplicit\nNeural Modules (XNMs), which advance beyond existing neural module networks\ntowards using scene graphs --- objects as nodes and the pairwise relationships\nas edges --- for explainable and explicit reasoning with structured knowledge.\nXNMs allow us to pay more attention to teach machines how to \"think\",\nregardless of what they \"look\". As we will show in the paper, by using scene\ngraphs as an inductive bias, 1) we can design XNMs in a concise and flexible\nfashion, i.e., XNMs merely consist of 4 meta-types, which significantly reduce\nthe number of parameters by 10 to 100 times, and 2) we can explicitly trace the\nreasoning-flow in terms of graph attentions. XNMs are so generic that they\nsupport a wide range of scene graph implementations with various qualities. For\nexample, when the graphs are detected perfectly, XNMs achieve 100% accuracy on\nboth CLEVR and CLEVR CoGenT, establishing an empirical performance upper-bound\nfor visual reasoning; when the graphs are noisily detected from real-world\nimages, XNMs are still robust to achieve a competitive 67.5% accuracy on\nVQAv2.0, surpassing the popular bag-of-objects attention models without graph\nstructures.","url_abs":"http://arxiv.org/abs/1812.01855v2","url_pdf":"http://arxiv.org/pdf/1812.01855v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"explainable-and-explicit-visual-reasoning","repo_url":"https://github.com/shijx12/XNM-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"explainable-and-explicit-visual-reasoning","repo_url":"https://github.com/shijx12/shijx12.github.io","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"},{"task_slug":"visual-reasoning","task_name":"Visual Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-clevr","task":"Visual Question Answering (VQA)","dataset":"CLEVR","model":"XNM-Det supervised","rank_in_archive_order":12,"of":15,"metrics":{"Accuracy":"97.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1812.01855","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1812.01855"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shijx12/XNM-Net","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shijx12/shijx12.github.io","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e6dbe2d3b6f0cc87","entry":"invert_dict","repo":"shijx12/XNM-Net","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/shijx12/XNM-Net/blob/HEAD/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6dbe2d3b6f0cc87"}},{"code_sha256_prefix":"6d12da501ae235a1","entry":"todevice","repo":"shijx12/XNM-Net","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/shijx12/XNM-Net/blob/HEAD/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d12da501ae235a1"}},{"code_sha256_prefix":"df94cf849791c151","entry":"convert_david_program_to_mine","repo":"shijx12/XNM-Net","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/shijx12/XNM-Net/blob/HEAD/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df94cf849791c151"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}