{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gap-a-graph-aware-language-model-framework","title":"GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation","arxiv_id":"2204.06674","date":"2022-04-13","proceeding":"COLING 2022 10","authors":["Anthony Colas","Mehrdad Alvandipour","Daisy Zhe Wang"],"abstract":"Recent improvements in KG-to-text generation are due to additional auxiliary pre-training tasks designed to give the fine-tune task a boost in performance. These tasks require extensive computational resources while only suggesting marginal improvements. Here, we demonstrate that by fusing graph-aware elements into existing pre-trained language models, we are able to outperform state-of-the-art models and close the gap imposed by additional pre-training tasks. We do so by proposing a mask structure to capture neighborhood information and a novel type encoder that adds a bias to the graph-attention weights depending on the connection type. Experiments on two KG-to-text benchmark datasets show our models are competitive while involving fewer parameters and no additional pre-training tasks. By formulating the problem as a framework, we can interchange the various proposed components and begin interpreting KG-to-text generative models based on the topological and type information found in a graph.","url_abs":"https://arxiv.org/abs/2204.06674v4","url_pdf":"https://arxiv.org/pdf/2204.06674v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gap-a-graph-aware-language-model-framework","repo_url":"https://github.com/acolas1/GAP_COLING2022","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-to-text-generation","task_name":"Data-to-Text Generation"},{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"kg-to-text","task_name":"KG-to-Text Generation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/kg-to-text-generation-on-eventnarrative","task":"KG-to-Text Generation","dataset":"EventNarrative","model":"GAP - Me,r+γ","rank_in_archive_order":1,"of":8,"metrics":{"BLEU":"35.08","BertScore":"93.38","METEOR":"27.5","ROUGE":"64.28"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-eventnarrative","task":"KG-to-Text Generation","dataset":"EventNarrative","model":"GAP - Me,re","rank_in_archive_order":2,"of":8,"metrics":{"BLEU":"34.02","METEOR":"26.93","ROUGE":"62.9"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-eventnarrative","task":"KG-to-Text Generation","dataset":"EventNarrative","model":"BART","rank_in_archive_order":4,"of":8,"metrics":{"BLEU":"31.38","BertScore":"93.12","METEOR":"26.68","ROUGE":"62.65"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-eventnarrative","task":"KG-to-Text Generation","dataset":"EventNarrative","model":"JointGT","rank_in_archive_order":5,"of":8,"metrics":{"BLEU":"31.19","BertScore":"93.68","METEOR":"26.58","ROUGE":"64.91"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-eventnarrative","task":"KG-to-Text Generation","dataset":"EventNarrative","model":"T5","rank_in_archive_order":8,"of":8,"metrics":{"BLEU":"12.8","BertScore":"89.59","METEOR":"22.77","ROUGE":"52.06"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"GAP - Me,r+γ","rank_in_archive_order":1,"of":13,"metrics":{"BLEU":"66.2","ROUGE":"76.36"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"JointGT (BART) - w/ JointGTPretrain","rank_in_archive_order":4,"of":13,"metrics":{"BLEU":"65.92","METEOR":"47.15","ROUGE":"76.1"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"JointGT (BART) - w/ BARTPretrain","rank_in_archive_order":5,"of":13,"metrics":{"BLEU":"64.6","METEOR":"46.77","ROUGE":"75.74"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"KGPT w/o pretrain","rank_in_archive_order":10,"of":13,"metrics":{"BLEU":"62.3","METEOR":"44.33","ROUGE":"73"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"GCN","rank_in_archive_order":12,"of":13,"metrics":{"BLEU":"60.8","METEOR":"42.76","ROUGE":"71.13"},"uses_additional_data":false},{"leaderboard":"/sota/kg-to-text-generation-on-webnlg-2-0","task":"KG-to-Text Generation","dataset":"WebNLG 2.0 (Unconstrained)","model":"GAP - Me,re","rank_in_archive_order":13,"of":13,"metrics":{"ROUGE":"76.22"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}