{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-practical-computational-power-of","title":"On the Practical Computational Power of Finite Precision RNNs for Language Recognition","arxiv_id":"1805.04908","date":"2018-05-13","proceeding":"ACL 2018 7","authors":["Gail Weiss","Yoav Goldberg","Eran Yahav"],"abstract":"While Recurrent Neural Networks (RNNs) are famously known to be Turing\ncomplete, this relies on infinite precision in the states and unbounded\ncomputation time. We consider the case of RNNs with finite precision whose\ncomputation time is linear in the input length. Under these limitations, we\nshow that different RNN variants have different computational power. In\nparticular, we show that the LSTM and the Elman-RNN with ReLU activation are\nstrictly stronger than the RNN with a squashing activation and the GRU. This is\nachieved because LSTMs and ReLU-RNNs can easily implement counting behavior. We\nshow empirically that the LSTM does indeed learn to effectively use the\ncounting mechanism.","url_abs":"http://arxiv.org/abs/1805.04908v1","url_pdf":"http://arxiv.org/pdf/1805.04908v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-practical-computational-power-of","repo_url":"https://github.com/hakunanatasha/counting_NN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"gru","method_name":"GRU"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.04908","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}