{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-binary-valued-gates-for-robust-lstm","title":"Towards Binary-Valued Gates for Robust LSTM Training","arxiv_id":"1806.02988","date":"2018-06-08","proceeding":"ICML 2018 7","authors":["Zhuohan Li","Di He","Fei Tian","Wei Chen","Tao Qin","Li-Wei Wang","Tie-Yan Liu"],"abstract":"Long Short-Term Memory (LSTM) is one of the most widely used recurrent\nstructures in sequence modeling. It aims to use gates to control information\nflow (e.g., whether to skip some information or not) in the recurrent\ncomputations, although its practical implementation based on soft gates only\npartially achieves this goal. In this paper, we propose a new way for LSTM\ntraining, which pushes the output values of the gates towards 0 or 1. By doing\nso, we can better control the information flow: the gates are mostly open or\nclosed, instead of in a middle state, which makes the results more\ninterpretable. Empirical studies show that (1) Although it seems that we\nrestrict the model capacity, there is no performance drop: we achieve better or\ncomparable performances due to its better generalization ability; (2) The\noutputs of gates are not sensitive to their inputs: we can easily compress the\nLSTM unit in multiple ways, e.g., low-rank approximation and low-precision\napproximation. The compressed models are even better than the baseline models\nwithout compression.","url_abs":"http://arxiv.org/abs/1806.02988v1","url_pdf":"http://arxiv.org/pdf/1806.02988v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-binary-valued-gates-for-robust-lstm","repo_url":"https://github.com/zhuohan123/g2-lstm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.02988","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.02988"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhuohan123/g2-lstm","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"ed3136aad02f89cb","entry":"param_init_lstm","repo":"zhuohan123/g2-lstm","repo_kind":"official","path":"machine-translation/libs/layers/stochastic_lstm.py","file_url":"https://github.com/zhuohan123/g2-lstm/blob/HEAD/machine-translation/libs/layers/stochastic_lstm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed3136aad02f89cb"}},{"code_sha256_prefix":"674575edead62d65","entry":"param_init_lstm_cond","repo":"zhuohan123/g2-lstm","repo_kind":"official","path":"machine-translation/libs/layers/stochastic_lstm.py","file_url":"https://github.com/zhuohan123/g2-lstm/blob/HEAD/machine-translation/libs/layers/stochastic_lstm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"674575edead62d65"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}