{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/information-aggregation-via-dynamic-routing","title":"Information Aggregation via Dynamic Routing for Sequence Encoding","arxiv_id":"1806.01501","date":"2018-06-05","proceeding":"COLING 2018 8","authors":["Jingjing Gong","Xipeng Qiu","Shaojing Wang","Xuanjing Huang"],"abstract":"While much progress has been made in how to encode a text sequence into a\nsequence of vectors, less attention has been paid to how to aggregate these\npreceding vectors (outputs of RNN/CNN) into fixed-size encoding vector.\nUsually, a simple max or average pooling is used, which is a bottom-up and\npassive way of aggregation and lack of guidance by task information. In this\npaper, we propose an aggregation mechanism to obtain a fixed-size encoding with\na dynamic routing policy. The dynamic routing policy is dynamically deciding\nthat what and how much information need be transferred from each word to the\nfinal encoding of the text sequence. Following the work of Capsule Network, we\ndesign two dynamic routing policies to aggregate the outputs of RNN/CNN\nencoding layer into a final encoding vector. Compared to the other aggregation\nmethods, dynamic routing can refine the messages according to the state of\nfinal encoding vector. Experimental results on five text classification tasks\nshow that our method outperforms other aggregating models by a significant\nmargin. Related source code is released on our github page.","url_abs":"http://arxiv.org/abs/1806.01501v1","url_pdf":"http://arxiv.org/pdf/1806.01501v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"information-aggregation-via-dynamic-routing","repo_url":"https://github.com/FudanNLP/Capsule4TextClassification","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"information-aggregation-via-dynamic-routing","repo_url":"https://github.com/michael-wzhu/dynamic_routing_pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"fixcaps","method_name":"Capsule Network"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-imdb","task":"Sentiment Analysis","dataset":"IMDb","model":"Standard DR-AGG","rank_in_archive_order":48,"of":49,"metrics":{"Accuracy":"45.1"},"uses_additional_data":true},{"leaderboard":"/sota/sentiment-analysis-on-imdb","task":"Sentiment Analysis","dataset":"IMDb","model":"Reverse DR-AGG","rank_in_archive_order":49,"of":49,"metrics":{"Accuracy":"44.5"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-sst-2-binary","task":"Sentiment Analysis","dataset":"SST-2 Binary classification","model":"Standard DR-AGG","rank_in_archive_order":71,"of":87,"metrics":{"Accuracy":"87.6"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-sst-2-binary","task":"Sentiment Analysis","dataset":"SST-2 Binary classification","model":"Reverse DR-AGG","rank_in_archive_order":73,"of":87,"metrics":{"Accuracy":"87.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.01501","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}