{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cheap-orthogonal-constraints-in-neural","title":"Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group","arxiv_id":"1901.08428","date":"2019-01-24","proceeding":null,"authors":["Mario Lezcano-Casado","David Martínez-Rubio"],"abstract":"We introduce a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theory through the exponential map. The parametrization transforms the constrained optimization problem into an unconstrained one over a Euclidean space, for which common first-order optimization methods can be used. The theoretical results presented are general enough to cover the special orthogonal group, the unitary group and, in general, any connected compact Lie group. We discuss how this and other parametrizations can be computed efficiently through an implementation trick, making numerically complex parametrizations usable at a negligible runtime cost in neural networks. In particular, we apply our results to RNNs with orthogonal recurrent weights, yielding a new architecture called expRNN. We demonstrate how our method constitutes a more robust approach to optimization with orthogonal constraints, showing faster, accurate, and more stable convergence in several tasks designed to test RNNs.","url_abs":"https://arxiv.org/abs/1901.08428v3","url_pdf":"https://arxiv.org/pdf/1901.08428v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cheap-orthogonal-constraints-in-neural","repo_url":"https://github.com/Lezcano/expRNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"cheap-orthogonal-constraints-in-neural","repo_url":"https://github.com/ndminhkhoi46/asRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"cheap-orthogonal-constraints-in-neural","repo_url":"https://github.com/Lezcano/geotorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.08428","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.08428"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ndminhkhoi46/asRNN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Lezcano/geotorch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Lezcano/expRNN","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"93fa2cac3cbd6302","entry":"get_parameters","repo":"Lezcano/expRNN","repo_kind":"official","path":"parametrization.py","file_url":"https://github.com/Lezcano/expRNN/blob/HEAD/parametrization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93fa2cac3cbd6302"}},{"code_sha256_prefix":"78009faf06e73385","entry":"get_lyap_exp","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/hurwitz.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/hurwitz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78009faf06e73385"}},{"code_sha256_prefix":"02208b0e975b51bb","entry":"inv_scaled_sigmoid","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/almostorthogonal.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/almostorthogonal.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02208b0e975b51bb"}},{"code_sha256_prefix":"c053e1ab6c41e08b","entry":"inv_softplus_epsilon","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/fixedrank.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/fixedrank.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c053e1ab6c41e08b"}},{"code_sha256_prefix":"6cbf25c3b9bfbff1","entry":"scaled_sigmoid","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/almostorthogonal.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/almostorthogonal.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cbf25c3b9bfbff1"}},{"code_sha256_prefix":"06458292351e1172","entry":"skew","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/constraints.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/constraints.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06458292351e1172"}},{"code_sha256_prefix":"1e7ed992f256ebcb","entry":"softplus_epsilon","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/fixedrank.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/fixedrank.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e7ed992f256ebcb"}},{"code_sha256_prefix":"9a20e3ddc0e62b11","entry":"sphere","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/constraints.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/constraints.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a20e3ddc0e62b11"}},{"code_sha256_prefix":"446a426848ccde8b","entry":"symmetric","repo":"Lezcano/geotorch","repo_kind":"listed","path":"geotorch/constraints.py","file_url":"https://github.com/Lezcano/geotorch/blob/HEAD/geotorch/constraints.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"446a426848ccde8b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}