{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/randomized-kaczmarz-methods-with-beyond","title":"Randomized Kaczmarz Methods with Beyond-Krylov Convergence","arxiv_id":"2501.11673","date":"2025-01-20","proceeding":null,"authors":["Michał Dereziński","Deanna Needell","Elizaveta Rebrova","Jiaming Yang"],"abstract":"Randomized Kaczmarz methods form a family of linear system solvers which converge by repeatedly projecting their iterates onto randomly sampled equations. While effective in some contexts, such as highly over-determined least squares, Kaczmarz methods are traditionally deemed secondary to Krylov subspace methods, since this latter family of solvers can exploit outliers in the input's singular value distribution to attain fast convergence on ill-conditioned systems. In this paper, we introduce Kaczmarz++, an accelerated randomized block Kaczmarz algorithm that exploits outlying singular values in the input to attain a fast Krylov-style convergence. Moreover, we show that Kaczmarz++ captures large outlying singular values provably faster than popular Krylov methods, for both over- and under-determined systems. We also develop an optimized variant for positive semidefinite systems, called CD++, demonstrating empirically that it is competitive in arithmetic operations with both CG and GMRES on a collection of benchmark problems. To attain these results, we introduce several novel algorithmic improvements to the Kaczmarz framework, including adaptive momentum acceleration, Tikhonov-regularized projections, and a memoization scheme for reusing information from previously sampled equation~blocks.","url_abs":"https://arxiv.org/abs/2501.11673v1","url_pdf":"https://arxiv.org/pdf/2501.11673v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"randomized-kaczmarz-methods-with-beyond","repo_url":"https://github.com/edwinyang7/kaczmarz-plusplus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"subspace-methods","task_name":"subspace methods"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.11673","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.11673"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/edwinyang7/kaczmarz-plusplus","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"47887edef57b909a","entry":"bernoulli_expo","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"kaczmarz.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/kaczmarz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47887edef57b909a"}},{"code_sha256_prefix":"fc55b38204b66af1","entry":"bernoulli_frac","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"kaczmarz.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/kaczmarz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc55b38204b66af1"}},{"code_sha256_prefix":"057f4371ccbf5847","entry":"compute_kernel","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"psd_flops.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/psd_flops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"057f4371ccbf5847"}},{"code_sha256_prefix":"494d1dfc48c2aeef","entry":"fht","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"494d1dfc48c2aeef"}},{"code_sha256_prefix":"c309582a7fb46f15","entry":"hadamard","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c309582a7fb46f15"}},{"code_sha256_prefix":"0945c71a377a43bc","entry":"inner_precondition","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"kaczmarz.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/kaczmarz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0945c71a377a43bc"}},{"code_sha256_prefix":"7661aeacfad625c2","entry":"load_dataset","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"kzpp_lsqr.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/kzpp_lsqr.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7661aeacfad625c2"}},{"code_sha256_prefix":"ae0524b73ecb0649","entry":"load_dataset","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"psd_flops.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/psd_flops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae0524b73ecb0649"}},{"code_sha256_prefix":"39022e01a51e9a39","entry":"rht","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39022e01a51e9a39"}},{"code_sha256_prefix":"1928c9c177832aa8","entry":"setup_system","repo":"edwinyang7/kaczmarz-plusplus","repo_kind":"official","path":"psd_flops.py","file_url":"https://github.com/edwinyang7/kaczmarz-plusplus/blob/HEAD/psd_flops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1928c9c177832aa8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}