{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-scale-gaussian-processes-via","title":"Large-Scale Gaussian Processes via Alternating Projection","arxiv_id":"2310.17137","date":"2023-10-26","proceeding":null,"authors":["Kaiwen Wu","Jonathan Wenger","Haydn Jones","Geoff Pleiss","Jacob R. Gardner"],"abstract":"Training and inference in Gaussian processes (GPs) require solving linear systems with $n\\times n$ kernel matrices. To address the prohibitive $\\mathcal{O}(n^3)$ time complexity, recent work has employed fast iterative methods, like conjugate gradients (CG). However, as datasets increase in magnitude, the kernel matrices become increasingly ill-conditioned and still require $\\mathcal{O}(n^2)$ space without partitioning. Thus, while CG increases the size of datasets GPs can be trained on, modern datasets reach scales beyond its applicability. In this work, we propose an iterative method which only accesses subblocks of the kernel matrix, effectively enabling mini-batching. Our algorithm, based on alternating projection, has $\\mathcal{O}(n)$ per-iteration time and space complexity, solving many of the practical challenges of scaling GPs to very large datasets. Theoretically, we prove the method enjoys linear convergence. Empirically, we demonstrate its fast convergence in practice and robustness to ill-conditioning. On large-scale benchmark datasets with up to four million data points, our approach accelerates GP training and inference by speed-up factors up to $27\\times$ and $72 \\times$, respectively, compared to CG.","url_abs":"https://arxiv.org/abs/2310.17137v2","url_pdf":"https://arxiv.org/pdf/2310.17137v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-scale-gaussian-processes-via","repo_url":"https://github.com/kayween/alternating-projection-for-gp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"}],"methods":[{"method_slug":"gps","method_name":"GPS"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.17137","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.17137"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kayween/alternating-projection-for-gp","reach":{"status":"ok"}}],"summary":{"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"57c20b2fda8b2e21","entry":"load_uci_data","repo":"kayween/alternating-projection-for-gp","repo_kind":"official","path":"data.py","file_url":"https://github.com/kayween/alternating-projection-for-gp/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57c20b2fda8b2e21"}},{"code_sha256_prefix":"6527788b76cf43d0","entry":"test","repo":"kayween/alternating-projection-for-gp","repo_kind":"official","path":"train_svgp.py","file_url":"https://github.com/kayween/alternating-projection-for-gp/blob/HEAD/train_svgp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6527788b76cf43d0"}},{"code_sha256_prefix":"3fd73c5ac27e0c52","entry":"train","repo":"kayween/alternating-projection-for-gp","repo_kind":"official","path":"utils.py","file_url":"https://github.com/kayween/alternating-projection-for-gp/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fd73c5ac27e0c52"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}