{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-abductive-reasoning","title":"Visual Abductive Reasoning","arxiv_id":"2203.14040","date":"2022-03-26","proceeding":"CVPR 2022 1","authors":["Chen Liang","Wenguan Wang","Tianfei Zhou","Yi Yang"],"abstract":"Abductive reasoning seeks the likeliest possible explanation for partial observations. Although abduction is frequently employed in human daily reasoning, it is rarely explored in computer vision literature. In this paper, we propose a new task and dataset, Visual Abductive Reasoning (VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations. Given an incomplete set of visual events, AI systems are required to not only describe what is observed, but also infer the hypothesis that can best explain the visual premise. Based on our large-scale VAR dataset, we devise a strong baseline model, Reasoner (causal-and-cascaded reasoning Transformer). First, to capture the causal structure of the observations, a contextualized directional position embedding strategy is adopted in the encoder, that yields discriminative representations for the premise and hypothesis. Then, multiple decoders are cascaded to generate and progressively refine the premise and hypothesis sentences. The prediction scores of the sentences are used to guide cross-sentence information flow in the cascaded reasoning procedure. Our VAR benchmarking results show that Reasoner surpasses many famous video-language models, while still being far behind human performance. This work is expected to foster future efforts in the reasoning-beyond-observation paradigm.","url_abs":"https://arxiv.org/abs/2203.14040v1","url_pdf":"https://arxiv.org/pdf/2203.14040v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-abductive-reasoning","repo_url":"https://github.com/leonnnop/var","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"visual-abductive-reasoning","task_name":"Visual Abductive Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.14040","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.14040"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leonnnop/var","reach":null}],"summary":{"ran":8,"ran_draft_wrong":1,"ran_fixture":2,"unverified":10},"by_repo_kind":{"official":{"samples":21,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"62fbcc09b795d0d9","entry":"BertEmbeddingsWithSentence","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62fbcc09b795d0d9"}},{"code_sha256_prefix":"8dcc67f7cdd708fe","entry":"BertLMPredictionHead","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dcc67f7cdd708fe"}},{"code_sha256_prefix":"af63ac23a3503cdf","entry":"BertPredictionHeadTransform","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af63ac23a3503cdf"}},{"code_sha256_prefix":"1ad7e9bae41c84c2","entry":"BertSelfAttention","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ad7e9bae41c84c2"}},{"code_sha256_prefix":"2a92c02716662f19","entry":"BertSelfAttentionSen","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a92c02716662f19"}},{"code_sha256_prefix":"e4821f9d2b6f5330","entry":"PositionEncoding","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4821f9d2b6f5330"}},{"code_sha256_prefix":"328e8c6cf27e19c8","entry":"RelativeTemEmbedding","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"328e8c6cf27e19c8"}},{"code_sha256_prefix":"d3d0c34b93dc9fec","entry":"TextEmbedding","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3d0c34b93dc9fec"}},{"code_sha256_prefix":"9fe227d9a54173f6","entry":"_concat_list_of_tensors","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fe227d9a54173f6"}},{"code_sha256_prefix":"ee2eb9135dec6e53","entry":"make_pad_shifted_mask","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee2eb9135dec6e53"}},{"code_sha256_prefix":"ed3b9085302e4898","entry":"make_shifted_mask","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed3b9085302e4898"}},{"code_sha256_prefix":"8a110cded88738a8","entry":"BertAttention","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a110cded88738a8"}},{"code_sha256_prefix":"860f9614a5fe7da7","entry":"BertEmbeddingsWithVideo","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"860f9614a5fe7da7"}},{"code_sha256_prefix":"2fbca415e4e6bf08","entry":"BertEncoder","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fbca415e4e6bf08"}},{"code_sha256_prefix":"5aeb53ac7ae0499a","entry":"BertEncoderSen","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5aeb53ac7ae0499a"}},{"code_sha256_prefix":"3d9f583a9ae50be3","entry":"BertEncoderUntied","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d9f583a9ae50be3"}},{"code_sha256_prefix":"5a8e31314ed696b6","entry":"BertLayer","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a8e31314ed696b6"}},{"code_sha256_prefix":"a677dcf23ed7fc33","entry":"BertLayerSen","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a677dcf23ed7fc33"}},{"code_sha256_prefix":"4d48738bd0f321b8","entry":"BertLayerUntied","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d48738bd0f321b8"}},{"code_sha256_prefix":"c9602cd24ba31174","entry":"Reasoner","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9602cd24ba31174"}},{"code_sha256_prefix":"0efbd684f43cf4f6","entry":"VideoEncodingTrans","repo":"leonnnop/var","repo_kind":"official","path":"src/model/mainmodel.py","file_url":"https://github.com/leonnnop/var/blob/HEAD/src/model/mainmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0efbd684f43cf4f6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}