{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ordered-neurons-integrating-tree-structures","title":"Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks","arxiv_id":"1810.09536","date":"2018-10-22","proceeding":"ICLR 2019 5","authors":["Yikang Shen","Shawn Tan","Alessandro Sordoni","Aaron Courville"],"abstract":"Natural language is hierarchically structured: smaller units (e.g., phrases) are nested within larger units (e.g., clauses). When a larger constituent ends, all of the smaller constituents that are nested within it must also be closed. While the standard LSTM architecture allows different neurons to track information at different time scales, it does not have an explicit bias towards modeling a hierarchy of constituents. This paper proposes to add such an inductive bias by ordering the neurons; a vector of master input and forget gates ensures that when a given neuron is updated, all the neurons that follow it in the ordering are also updated. Our novel recurrent architecture, ordered neurons LSTM (ON-LSTM), achieves good performance on four different tasks: language modeling, unsupervised parsing, targeted syntactic evaluation, and logical inference.","url_abs":"https://arxiv.org/abs/1810.09536v6","url_pdf":"https://arxiv.org/pdf/1810.09536v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/yikangshen/Ordered-Neurons","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/IanTheColder/ONLSTM-analysis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/TieDanCuihua/Hierarchical-Attention-Networks-for-Document-Classification-Tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/TieDanCuihua/ORDERED-NEURONS-INTEGRATING-TREE-STRUCTURES-INTO-RECURRENT-NEURAL-NETWORKS--tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/billptw/fasttrees","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/whull/ONLSTM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"ordered-neurons-integrating-tree-structures","repo_url":"https://github.com/zi-lin/on-lstm-tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"constituency-grammar-induction","task_name":"Constituency Grammar Induction"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/constituency-grammar-induction-on-ptb","task":"Constituency Grammar Induction","dataset":"PTB Diagnostic ECG Database","model":"ON-LSTM (tuned)","rank_in_archive_order":19,"of":24,"metrics":{"Max F1 (WSJ)":"50.0","Mean F1 (WSJ)":"48.1"},"uses_additional_data":false},{"leaderboard":"/sota/constituency-grammar-induction-on-ptb","task":"Constituency Grammar Induction","dataset":"PTB Diagnostic ECG Database","model":"ON-LSTM","rank_in_archive_order":21,"of":24,"metrics":{"Max F1 (WSJ)":"49.4","Max F1 (WSJ10)":"66.8","Mean F1 (WSJ)":"47.7","Mean F1 (WSJ10)":"65.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.09536","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1810.09536"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yikangshen/Ordered-Neurons","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/billptw/fasttrees","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IanTheColder/ONLSTM-analysis","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zi-lin/on-lstm-tensorflow","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/whull/ONLSTM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TieDanCuihua/ORDERED-NEURONS-INTEGRATING-TREE-STRUCTURES-INTO-RECURRENT-NEURAL-NETWORKS--tensorflow","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TieDanCuihua/Hierarchical-Attention-Networks-for-Document-Classification-Tensorflow","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"ran_honours":1,"unverified":9},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1},"listed":{"samples":10,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2cdc4ed134986036","entry":"build_tree","repo":"yikangshen/Ordered-Neurons","repo_kind":"official","path":"test_phrase_grammar.py","file_url":"https://github.com/yikangshen/Ordered-Neurons/blob/HEAD/test_phrase_grammar.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2cdc4ed134986036"}},{"code_sha256_prefix":"4b59eaae917b5a2d","entry":"cumsoftmax","repo":"yikangshen/Ordered-Neurons","repo_kind":"official","path":"ON_LSTM.py","file_url":"https://github.com/yikangshen/Ordered-Neurons/blob/HEAD/ON_LSTM.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"4b59eaae917b5a2d"}},{"code_sha256_prefix":"4372bb4533fd1936","entry":"repackage_hidden","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"4372bb4533fd1936"}},{"code_sha256_prefix":"bef0350ba4dc9a48","entry":"balance","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"parse_comparison.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/parse_comparison.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"bef0350ba4dc9a48"}},{"code_sha256_prefix":"6bd10c3dd56c2b8c","entry":"batchify","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6bd10c3dd56c2b8c"}},{"code_sha256_prefix":"71364ee53326b360","entry":"cumsoftmax","repo":"zi-lin/on-lstm-tensorflow","repo_kind":"listed","path":"on_lstm_layer.py","file_url":"https://github.com/zi-lin/on-lstm-tensorflow/blob/HEAD/on_lstm_layer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71364ee53326b360"}},{"code_sha256_prefix":"62d148258bdd054f","entry":"embedded_dropout","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"embed_regularize.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/embed_regularize.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"62d148258bdd054f"}},{"code_sha256_prefix":"6f3c8e0b36d76af7","entry":"get_batch","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6f3c8e0b36d76af7"}},{"code_sha256_prefix":"7e3df128e06d7869","entry":"parenthesize","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"parse_by_baseline.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/parse_by_baseline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"7e3df128e06d7869"}},{"code_sha256_prefix":"31b75ccedc2bc209","entry":"randomize_split","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"parse_by_baseline.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/parse_by_baseline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"31b75ccedc2bc209"}},{"code_sha256_prefix":"219eb921a74ea62c","entry":"roundup2","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"parse_comparison.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/parse_comparison.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"219eb921a74ea62c"}},{"code_sha256_prefix":"ed1c4f48a3e8e92e","entry":"spaceify","repo":"IanTheColder/ONLSTM-analysis","repo_kind":"listed","path":"parse_comparison.py","file_url":"https://github.com/IanTheColder/ONLSTM-analysis/blob/HEAD/parse_comparison.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ed1c4f48a3e8e92e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}