{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/monotonic-chunkwise-attention","title":"Monotonic Chunkwise Attention","arxiv_id":"1712.05382","date":"2017-12-14","proceeding":null,"authors":["Chung-Cheng Chiu","Colin Raffel"],"abstract":"Sequence-to-sequence models with soft attention have been successfully\napplied to a wide variety of problems, but their decoding process incurs a\nquadratic time and space cost and is inapplicable to real-time sequence\ntransduction. To address these issues, we propose Monotonic Chunkwise Attention\n(MoChA), which adaptively splits the input sequence into small chunks over\nwhich soft attention is computed. We show that models utilizing MoChA can be\ntrained efficiently with standard backpropagation while allowing online and\nlinear-time decoding at test time. When applied to online speech recognition,\nwe obtain state-of-the-art results and match the performance of a model using\nan offline soft attention mechanism. In document summarization experiments\nwhere we do not expect monotonic alignments, we show significantly improved\nperformance compared to a baseline monotonic attention-based model.","url_abs":"http://arxiv.org/abs/1712.05382v2","url_pdf":"http://arxiv.org/pdf/1712.05382v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"monotonic-chunkwise-attention","repo_url":"https://github.com/craffel/mocha","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"document-summarization","task_name":"Document Summarization"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1712.05382","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}