{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimization-without-retraction-on-the-random","title":"Optimization without Retraction on the Random Generalized Stiefel Manifold","arxiv_id":"2405.01702","date":"2024-05-02","proceeding":null,"authors":["Simon Vary","Pierre Ablin","Bin Gao","P. -A. Absil"],"abstract":"Optimization over the set of matrices $X$ that satisfy $X^\\top B X = I_p$, referred to as the generalized Stiefel manifold, appears in many applications involving sampled covariance matrices such as the canonical correlation analysis (CCA), independent component analysis (ICA), and the generalized eigenvalue problem (GEVP). Solving these problems is typically done by iterative methods that require a fully formed $B$. We propose a cheap stochastic iterative method that solves the optimization problem while having access only to random estimates of $B$. Our method does not enforce the constraint in every iteration; instead, it produces iterations that converge to critical points on the generalized Stiefel manifold defined in expectation. The method has lower per-iteration cost, requires only matrix multiplications, and has the same convergence rates as its Riemannian optimization counterparts that require the full matrix $B$. Experiments demonstrate its effectiveness in various machine learning applications involving generalized orthogonality constraints, including CCA, ICA, and the GEVP.","url_abs":"https://arxiv.org/abs/2405.01702v3","url_pdf":"https://arxiv.org/pdf/2405.01702v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimization-without-retraction-on-the-random","repo_url":"https://github.com/simonvary/landing-generalized-stiefel","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"riemannian-optimization","task_name":"Riemannian optimization"}],"methods":[{"method_slug":"ica","method_name":"ICA"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.01702","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.01702"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/simonvary/landing-generalized-stiefel","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"119e079dc606255a","entry":"LandingGeneralizedStiefel","repo":"simonvary/landing-generalized-stiefel","repo_kind":"official","path":"solvers/landing_generalized_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-generalized-stiefel/blob/HEAD/solvers/landing_generalized_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"119e079dc606255a"}},{"code_sha256_prefix":"2fe9c13adc810cd7","entry":"sylvester","repo":"simonvary/landing-generalized-stiefel","repo_kind":"official","path":"solvers/landing_generalized_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-generalized-stiefel/blob/HEAD/solvers/landing_generalized_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2fe9c13adc810cd7"}},{"code_sha256_prefix":"ebf360d0ee016e41","entry":"symm","repo":"simonvary/landing-generalized-stiefel","repo_kind":"official","path":"solvers/landing_generalized_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-generalized-stiefel/blob/HEAD/solvers/landing_generalized_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ebf360d0ee016e41"}},{"code_sha256_prefix":"eb5da0fcfb6fc7ef","entry":"symm","repo":"simonvary/landing-generalized-stiefel","repo_kind":"official","path":"solvers/landing_generalized_stiefel/optimizer.py","file_url":"https://github.com/simonvary/landing-generalized-stiefel/blob/HEAD/solvers/landing_generalized_stiefel/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb5da0fcfb6fc7ef"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}