{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/infeasible-deterministic-stochastic-and","title":"Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints","arxiv_id":"2303.16510","date":"2023-03-29","proceeding":null,"authors":["Pierre Ablin","Simon Vary","Bin Gao","P. -A. Absil"],"abstract":"Orthogonality constraints naturally appear in many machine learning problems, from principal component analysis to robust neural network training. They are usually solved using Riemannian optimization algorithms, which minimize the objective function while enforcing the constraint. However, enforcing the orthogonality constraint can be the most time-consuming operation in such algorithms. Recently, Ablin & Peyr\\'e (2022) proposed the landing algorithm, a method with cheap iterations that does not enforce the orthogonality constraints but is attracted towards the manifold in a smooth manner. This article provides new practical and theoretical developments for the landing algorithm. First, the method is extended to the Stiefel manifold, the set of rectangular orthogonal matrices. We also consider stochastic and variance reduction algorithms when the cost function is an average of many functions. We demonstrate that all these methods have the same rate of convergence as their Riemannian counterparts that exactly enforce the constraint, and converge to the manifold. Finally, our experiments demonstrate the promise of our approach to an array of machine-learning problems that involve orthogonality constraints.","url_abs":"https://arxiv.org/abs/2303.16510v2","url_pdf":"https://arxiv.org/pdf/2303.16510v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"infeasible-deterministic-stochastic-and","repo_url":"https://github.com/simonvary/landing-stiefel","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"riemannian-optimization","task_name":"Riemannian optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.16510","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.16510"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/simonvary/landing-stiefel","reach":null}],"summary":{"ran_draft_wrong":6},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"7b4edfdb9154ad16","entry":"reshape_conv2d_weight","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"landing_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/landing_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b4edfdb9154ad16"}},{"code_sha256_prefix":"4c7e0a2af22f059b","entry":"reshape_conv2d_weight_back","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"landing_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/landing_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c7e0a2af22f059b"}},{"code_sha256_prefix":"395dcadd2d4cddfe","entry":"scheduler_function","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"experiments/1_pca.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/experiments/1_pca.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"395dcadd2d4cddfe"}},{"code_sha256_prefix":"8bccc665ff5050bd","entry":"scheduler_function","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"experiments/2_cifar10_resnet.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/experiments/2_cifar10_resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8bccc665ff5050bd"}},{"code_sha256_prefix":"5270a1ba3ee0fa81","entry":"scheduler_function_geotorch","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"experiments/2_cifar10_resnet.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/experiments/2_cifar10_resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5270a1ba3ee0fa81"}},{"code_sha256_prefix":"efd158c1fc67fb7a","entry":"scheduler_function_geotorch2","repo":"simonvary/landing-stiefel","repo_kind":"official","path":"experiments/2_cifar10_resnet.py","file_url":"https://github.com/simonvary/landing-stiefel/blob/HEAD/experiments/2_cifar10_resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"efd158c1fc67fb7a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}