{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/video-as-conditional-graph-hierarchy-for","title":"Video as Conditional Graph Hierarchy for Multi-Granular Question Answering","arxiv_id":"2112.06197","date":"2021-12-12","proceeding":null,"authors":["Junbin Xiao","Angela Yao","Zhiyuan Liu","Yicong Li","Wei Ji","Tat-Seng Chua"],"abstract":"Video question answering requires the models to understand and reason about both the complex video and language data to correctly derive the answers. Existing efforts have been focused on designing sophisticated cross-modal interactions to fuse the information from two modalities, while encoding the video and question holistically as frame and word sequences. Despite their success, these methods are essentially revolving around the sequential nature of video- and question-contents, providing little insight to the problem of question-answering and lacking interpretability as well. In this work, we argue that while video is presented in frame sequence, the visual elements (e.g., objects, actions, activities and events) are not sequential but rather hierarchical in semantic space. To align with the multi-granular essence of linguistic concepts in language queries, we propose to model video as a conditional graph hierarchy which weaves together visual facts of different granularity in a level-wise manner, with the guidance of corresponding textual cues. Despite the simplicity, our extensive experiments demonstrate the superiority of such conditional hierarchical graph architecture, with clear performance improvements over prior methods and also better generalization across different type of questions. Further analyses also demonstrate the model's reliability as it shows meaningful visual-textual evidences for the predicted answers.","url_abs":"https://arxiv.org/abs/2112.06197v2","url_pdf":"https://arxiv.org/pdf/2112.06197v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"video-as-conditional-graph-hierarchy-for","repo_url":"https://github.com/doc-doc/hqga","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"video-question-answering","task_name":"Video Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-question-answering-on-intentqa","task":"Video Question Answering","dataset":"IntentQA","model":"HQGA","rank_in_archive_order":6,"of":6,"metrics":{"Accuarcy":"47.7","CH":"54.3","CW":"48.2","TP&TN":"41.7"},"uses_additional_data":false},{"leaderboard":"/sota/video-question-answering-on-next-qa","task":"Video Question Answering","dataset":"NExT-QA","model":"HQGA","rank_in_archive_order":47,"of":47,"metrics":{"Accuracy":"51.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.06197","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.06197"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/doc-doc/hqga","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":5},"by_repo_kind":{"official":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"69664111108e8701","entry":"positionalencoding2d","repo":"doc-doc/hqga","repo_kind":"official","path":"networks/PosEmbed.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/networks/PosEmbed.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69664111108e8701"}},{"code_sha256_prefix":"e649c25c10f968ce","entry":"load_file","repo":"doc-doc/hqga","repo_kind":"official","path":"utils.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e649c25c10f968ce"}},{"code_sha256_prefix":"d90d68789e4723ae","entry":"pkload","repo":"doc-doc/hqga","repo_kind":"official","path":"utils.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d90d68789e4723ae"}},{"code_sha256_prefix":"3c3fb01a914c2038","entry":"positionalencoding1d","repo":"doc-doc/hqga","repo_kind":"official","path":"networks/PosEmbed.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/networks/PosEmbed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c3fb01a914c2038"}},{"code_sha256_prefix":"2c170d86103f5cba","entry":"to_device","repo":"doc-doc/hqga","repo_kind":"official","path":"utils.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c170d86103f5cba"}},{"code_sha256_prefix":"663e53e83acb4524","entry":"unk_num","repo":"doc-doc/hqga","repo_kind":"official","path":"videoqa.py","file_url":"https://github.com/doc-doc/hqga/blob/HEAD/videoqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"663e53e83acb4524"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}