{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-recurrent-binaryternary-weights","title":"Learning Recurrent Binary/Ternary Weights","arxiv_id":"1809.11086","date":"2018-09-28","proceeding":"ICLR 2019 5","authors":["Arash Ardakani","Zhengyun Ji","Sean C. Smithson","Brett H. Meyer","Warren J. Gross"],"abstract":"Recurrent neural networks (RNNs) have shown excellent performance in\nprocessing sequence data. However, they are both complex and memory intensive\ndue to their recursive nature. These limitations make RNNs difficult to embed\non mobile devices requiring real-time processes with limited hardware\nresources. To address the above issues, we introduce a method that can learn\nbinary and ternary weights during the training phase to facilitate hardware\nimplementations of RNNs. As a result, using this approach replaces all\nmultiply-accumulate operations by simple accumulations, bringing significant\nbenefits to custom hardware in terms of silicon area and power consumption. On\nthe software side, we evaluate the performance (in terms of accuracy) of our\nmethod using long short-term memories (LSTMs) on various sequential models\nincluding sequence classification and language modeling. We demonstrate that\nour method achieves competitive results on the aforementioned tasks while using\nbinary/ternary weights during the runtime. On the hardware side, we present\ncustom hardware for accelerating the recurrent computations of LSTMs with\nbinary/ternary weights. Ultimately, we show that LSTMs with binary/ternary\nweights can achieve up to 12x memory saving and 10x inference speedup compared\nto the full-precision implementation on an ASIC platform.","url_abs":"http://arxiv.org/abs/1809.11086v2","url_pdf":"http://arxiv.org/pdf/1809.11086v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-recurrent-binaryternary-weights","repo_url":"https://github.com/arashardakani/Learning-Recurrent-Binary-Ternary-Weights","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}