{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/convolutional-sequence-modeling-revisited","title":"Convolutional Sequence Modeling Revisited","arxiv_id":null,"date":"2018-01-01","proceeding":"ICLR 2018 1","authors":["Shaojie Bai","J. Zico Kolter","Vladlen Koltun"],"abstract":"This paper revisits the problem of sequence modeling using convolutional \narchitectures.  Although both convolutional and recurrent architectures have a\nlong history in sequence prediction, the current \"default\" mindset in much of\nthe deep learning community is that generic sequence modeling is best handled\nusing recurrent networks.  The goal of this paper is to question this assumption. \nSpecifically, we consider a simple generic temporal convolution network (TCN),\nwhich adopts features from modern ConvNet architectures such as a dilations and \nresidual connections.  We show that on a variety of sequence modeling tasks,\nincluding many frequently used as benchmarks for evaluating recurrent networks,\nthe TCN outperforms baseline RNN methods (LSTMs, GRUs, and vanilla RNNs) and\nsometimes even highly specialized approaches.  We further show that the\npotential \"infinite memory\" advantage that RNNs have over TCNs is largely\nabsent in practice: TCNs indeed exhibit longer effective history sizes than their \nrecurrent counterparts.   As a whole, we argue that it may be time to (re)consider \nConvNets as the default \"go to\" architecture for sequence modeling.","url_abs":"https://openreview.net/forum?id=rk8wKk-R-","url_pdf":"https://openreview.net/pdf?id=rk8wKk-R-","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/language-modelling-on-wikitext-103","task":"Language Modelling","dataset":"WikiText-103","model":"Temporal CNN","rank_in_archive_order":84,"of":89,"metrics":{"Test perplexity":"45.2","Validation perplexity":"-"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}