{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fusionnet-fusing-via-fully-aware-attention","title":"FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension","arxiv_id":"1711.07341","date":"2017-11-16","proceeding":"ICLR 2018 1","authors":["Hsin-Yuan Huang","Chenguang Zhu","Yelong Shen","Weizhu Chen"],"abstract":"This paper introduces a new neural structure called FusionNet, which extends\nexisting attention approaches from three perspectives. First, it puts forward a\nnovel concept of \"history of word\" to characterize attention information from\nthe lowest word-level embedding up to the highest semantic-level\nrepresentation. Second, it introduces an improved attention scoring function\nthat better utilizes the \"history of word\" concept. Third, it proposes a\nfully-aware multi-level attention mechanism to capture the complete information\nin one text (such as a question) and exploit it in its counterpart (such as\ncontext or passage) layer by layer. We apply FusionNet to the Stanford Question\nAnswering Dataset (SQuAD) and it achieves the first position for both single\nand ensemble model on the official SQuAD leaderboard at the time of writing\n(Oct. 4th, 2017). Meanwhile, we verify the generalization of FusionNet with two\nadversarial SQuAD datasets and it sets up the new state-of-the-art on both\ndatasets: on AddSent, FusionNet increases the best F1 metric from 46.6% to\n51.4%; on AddOneSent, FusionNet boosts the best F1 metric from 56.0% to 60.7%.","url_abs":"http://arxiv.org/abs/1711.07341v2","url_pdf":"http://arxiv.org/pdf/1711.07341v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fusionnet-fusing-via-fully-aware-attention","repo_url":"https://github.com/momohuang/FusionNet-NLI","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"fusionnet-fusing-via-fully-aware-attention","repo_url":"https://github.com/felixgwu/FastFusionNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"fusionnet-fusing-via-fully-aware-attention","repo_url":"https://github.com/yellowpsyduck/OccamFusionNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-squad11","task":"Question Answering","dataset":"SQuAD1.1","model":"FusionNet (ensemble)","rank_in_archive_order":80,"of":213,"metrics":{"EM":"78.978","F1":"86.016"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad11","task":"Question Answering","dataset":"SQuAD1.1","model":"FusionNet (single model)","rank_in_archive_order":115,"of":213,"metrics":{"EM":"75.968","F1":"83.900"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad11-dev","task":"Question Answering","dataset":"SQuAD1.1 dev","model":"FusionNet","rank_in_archive_order":26,"of":55,"metrics":{"EM":"75.3","F1":"83.6"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad20","task":"Question Answering","dataset":"SQuAD2.0","model":"FusionNet++ (ensemble)","rank_in_archive_order":252,"of":286,"metrics":{"EM":"70.300","F1":"72.484"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.07341","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}