{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/document-modeling-with-graph-attention","title":"Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension","arxiv_id":"2005.05806","date":"2020-05-12","proceeding":"ACL 2020 6","authors":["Bo Zheng","Haoyang Wen","Yaobo Liang","Nan Duan","Wanxiang Che","Daxin Jiang","Ming Zhou","Ting Liu"],"abstract":"Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer). Despite the effectiveness of existing methods on this benchmark, they treat these two sub-tasks individually during training while ignoring their dependencies. To address this issue, we present a novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature, which are different levels of granularity: documents, paragraphs, sentences, and tokens. We utilize graph attention networks to obtain different levels of representations so that they can be learned simultaneously. The long and short answers can be extracted from paragraph-level representation and token-level representation, respectively. In this way, we can model the dependencies between the two-grained answers to provide evidence for each other. We jointly train the two sub-tasks, and our experiments show that our approach significantly outperforms previous systems at both long and short answer criteria.","url_abs":"https://arxiv.org/abs/2005.05806v2","url_pdf":"https://arxiv.org/pdf/2005.05806v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"document-modeling-with-graph-attention","repo_url":"https://github.com/DancingSoul/NQ_BERT-DM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"machine-reading-comprehension","task_name":"Machine Reading Comprehension"},{"task_slug":"natural-questions","task_name":"Natural Questions"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.05806","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.05806"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/DancingSoul/NQ_BERT-DM","reach":null}],"summary":{"ran":8,"unverified":8},"by_repo_kind":{"official":{"samples":16,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"eb406f6707a00ab5","entry":"AttentionOutputLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb406f6707a00ab5"}},{"code_sha256_prefix":"5ee54f32fd891520","entry":"Config","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ee54f32fd891520"}},{"code_sha256_prefix":"285c454ad1551507","entry":"Embeddings","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"285c454ad1551507"}},{"code_sha256_prefix":"e44a2df8a877baba","entry":"Initializer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e44a2df8a877baba"}},{"code_sha256_prefix":"8421424641c58fe2","entry":"IntegrationLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8421424641c58fe2"}},{"code_sha256_prefix":"5293729cc460256c","entry":"IntermediateLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5293729cc460256c"}},{"code_sha256_prefix":"7d637db54b1f2f78","entry":"LayerNorm","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d637db54b1f2f78"}},{"code_sha256_prefix":"4ef4d15f103a2b67","entry":"OutputLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ef4d15f103a2b67"}},{"code_sha256_prefix":"328a352304d452f0","entry":"EdgeType","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"328a352304d452f0"}},{"code_sha256_prefix":"61cf9ac60bc28ea1","entry":"Encoder","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61cf9ac60bc28ea1"}},{"code_sha256_prefix":"04a6fb299fea5a82","entry":"EncoderLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04a6fb299fea5a82"}},{"code_sha256_prefix":"61af68f9191fc0d0","entry":"GraphAttention","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61af68f9191fc0d0"}},{"code_sha256_prefix":"472f4c28f8396e93","entry":"GraphAttentionLayer","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"472f4c28f8396e93"}},{"code_sha256_prefix":"8d472f7368f335e3","entry":"GraphEncoder","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d472f7368f335e3"}},{"code_sha256_prefix":"3f23ae7e8ac9411b","entry":"NodePosition","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f23ae7e8ac9411b"}},{"code_sha256_prefix":"7662a878ed2c7d57","entry":"PreTrainedModel","repo":"DancingSoul/NQ_BERT-DM","repo_kind":"official","path":"modules/graph_encoder.py","file_url":"https://github.com/DancingSoul/NQ_BERT-DM/blob/HEAD/modules/graph_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7662a878ed2c7d57"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}