{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/qa-gnn-reasoning-with-language-models-and","title":"QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering","arxiv_id":"2104.06378","date":"2021-04-13","proceeding":"NAACL 2021 4","authors":["Michihiro Yasunaga","Hongyu Ren","Antoine Bosselut","Percy Liang","Jure Leskovec"],"abstract":"The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify relevant knowledge from large KGs, and (ii) perform joint reasoning over the QA context and KG. In this work, we propose a new model, QA-GNN, which addresses the above challenges through two key innovations: (i) relevance scoring, where we use LMs to estimate the importance of KG nodes relative to the given QA context, and (ii) joint reasoning, where we connect the QA context and KG to form a joint graph, and mutually update their representations through graph neural networks. We evaluate our model on QA benchmarks in the commonsense (CommonsenseQA, OpenBookQA) and biomedical (MedQA-USMLE) domains. QA-GNN outperforms existing LM and LM+KG models, and exhibits capabilities to perform interpretable and structured reasoning, e.g., correctly handling negation in questions.","url_abs":"https://arxiv.org/abs/2104.06378v5","url_pdf":"https://arxiv.org/pdf/2104.06378v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://github.com/michiyasunaga/qagnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://worksheets.codalab.org/worksheets/0xf215deb05edf44a2ac353c711f52a25f","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://github.com/CMSC470-Team/Model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://github.com/HaochenLiu2000/QAP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://github.com/rucaibox/safe","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"qa-gnn-reasoning-with-language-models-and","repo_url":"https://github.com/tobias-opsahl/fact-or-fiction","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":null,"task_name":"MedQA"},{"task_slug":"multi-hop-question-answering","task_name":"Multi-hop Question Answering"},{"task_slug":"negation","task_name":"Negation"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"riddle-sense","task_name":"Riddle Sense"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gat","method_name":"GAT"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/common-sense-reasoning-on-commonsenseqa","task":"Common Sense Reasoning","dataset":"CommonsenseQA","model":"QA-GNN","rank_in_archive_order":13,"of":38,"metrics":{"Accuracy":"76.1"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-openbookqa","task":"Question Answering","dataset":"OpenBookQA","model":"AristoRoBERTa + QA-GNN","rank_in_archive_order":20,"of":45,"metrics":{"Accuracy":"82.8"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-openbookqa","task":"Question Answering","dataset":"OpenBookQA","model":"QA-GNN","rank_in_archive_order":21,"of":45,"metrics":{"Accuracy":"82.8"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-openbookqa","task":"Question Answering","dataset":"OpenBookQA","model":"AristoRoBERTa","rank_in_archive_order":25,"of":45,"metrics":{"Accuracy":"77.8"},"uses_additional_data":false},{"leaderboard":"/sota/riddle-sense-on-riddle-sense","task":"Riddle Sense","dataset":"RiddleSense","model":"QAGNN","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy (%)":"67"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.06378","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.06378"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HaochenLiu2000/QAP","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rucaibox/safe","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michiyasunaga/qagnn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://worksheets.codalab.org/worksheets/0xf215deb05edf44a2ac353c711f52a25f","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CMSC470-Team/Model","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tobias-opsahl/fact-or-fiction","reach":null}],"summary":{"ran_honours":1,"unverified":24},"by_repo_kind":{"official":{"samples":21,"ran":1,"repositories":1},"listed":{"samples":4,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"b75e9c793d2a286a","entry":"gelu","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/layers.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/layers.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b75e9c793d2a286a"}},{"code_sha256_prefix":"2d38a4d448347c3c","entry":"CustomGraphTransformerLayer","repo":"HaochenLiu2000/QAP","repo_kind":"listed","path":"medqa/train_qap.py","file_url":"https://github.com/HaochenLiu2000/QAP/blob/HEAD/medqa/train_qap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2d38a4d448347c3c"}},{"code_sha256_prefix":"941efc4033b96f8c","entry":"GraphTransformer","repo":"HaochenLiu2000/QAP","repo_kind":"listed","path":"medqa/train_qap.py","file_url":"https://github.com/HaochenLiu2000/QAP/blob/HEAD/medqa/train_qap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"941efc4033b96f8c"}},{"code_sha256_prefix":"f82bdec01f779733","entry":"QAGNN","repo":"tobias-opsahl/fact-or-fiction","repo_kind":"listed","path":"models.py","file_url":"https://github.com/tobias-opsahl/fact-or-fiction/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f82bdec01f779733"}},{"code_sha256_prefix":"e6a99b8687aab99c","entry":"concepts2adj","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/graph.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6a99b8687aab99c"}},{"code_sha256_prefix":"be8c35c9c1d52b66","entry":"construct_graph","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/conceptnet.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/conceptnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be8c35c9c1d52b66"}},{"code_sha256_prefix":"15baaf34a0e42ec6","entry":"convert_qajson_to_entailment","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/convert_csqa.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/convert_csqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15baaf34a0e42ec6"}},{"code_sha256_prefix":"e4b000b919d28344","entry":"create_hypothesis","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/convert_csqa.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/convert_csqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4b000b919d28344"}},{"code_sha256_prefix":"345456d568545c8e","entry":"create_output_dict","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/convert_obqa.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/convert_obqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"345456d568545c8e"}},{"code_sha256_prefix":"5f4eefd99c11e698","entry":"del_pos","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/conceptnet.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/conceptnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f4eefd99c11e698"}},{"code_sha256_prefix":"866c784c24ba2fa8","entry":"dropout_mask","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/layers.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"866c784c24ba2fa8"}},{"code_sha256_prefix":"086fa0038784db8c","entry":"evaluate_accuracy","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"qagnn.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/qagnn.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"086fa0038784db8c"}},{"code_sha256_prefix":"96ec8d520ae857f1","entry":"generate_adj_matrix_per_inst","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/graph.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96ec8d520ae857f1"}},{"code_sha256_prefix":"216445b74badf787","entry":"get_bert_model","repo":"tobias-opsahl/fact-or-fiction","repo_kind":"listed","path":"models.py","file_url":"https://github.com/tobias-opsahl/fact-or-fiction/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"216445b74badf787"}},{"code_sha256_prefix":"a9a454bf4414e3fe","entry":"get_fitb_from_question","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/convert_csqa.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/convert_csqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9a454bf4414e3fe"}},{"code_sha256_prefix":"9dcf00b849de8977","entry":"glove_init","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/conceptnet.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/conceptnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9dcf00b849de8977"}},{"code_sha256_prefix":"5ff1624579da2bdc","entry":"lemmatize","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/grounding.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/grounding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ff1624579da2bdc"}},{"code_sha256_prefix":"b5768cf5e27d534d","entry":"load_bert_xlnet_roberta_input_tensors","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5768cf5e27d534d"}},{"code_sha256_prefix":"2f104114280df441","entry":"load_cpnet_vocab","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/grounding.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/grounding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f104114280df441"}},{"code_sha256_prefix":"e633666b83ca73b6","entry":"load_gpt_input_tensors","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e633666b83ca73b6"}},{"code_sha256_prefix":"16bd49eb2e3b8970","entry":"load_sparse_adj_data_with_contextnode","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16bd49eb2e3b8970"}},{"code_sha256_prefix":"df5e0b02401bf941","entry":"make_one_hot","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"modeling/modeling_qagnn.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/modeling/modeling_qagnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df5e0b02401bf941"}},{"code_sha256_prefix":"3841430a72ff67fa","entry":"masked_softmax","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/layers.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3841430a72ff67fa"}},{"code_sha256_prefix":"5edf23a7957ba803","entry":"normalize_sparse_adj","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/maths.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/maths.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5edf23a7957ba803"}},{"code_sha256_prefix":"66135965fe362f47","entry":"plain_graph_generation","repo":"michiyasunaga/qagnn","repo_kind":"official","path":"utils/graph.py","file_url":"https://github.com/michiyasunaga/qagnn/blob/HEAD/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66135965fe362f47"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}