{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/supervised-kernel-thinning","title":"Supervised Kernel Thinning","arxiv_id":"2410.13749","date":"2024-10-17","proceeding":null,"authors":["Albert Gong","Kyuseong Choi","Raaz Dwivedi"],"abstract":"The kernel thinning algorithm of Dwivedi & Mackey (2024) provides a better-than-i.i.d. compression of a generic set of points. By generating high-fidelity coresets of size significantly smaller than the input points, KT is known to speed up unsupervised tasks like Monte Carlo integration, uncertainty quantification, and non-parametric hypothesis testing, with minimal loss in statistical accuracy. In this work, we generalize the KT algorithm to speed up supervised learning problems involving kernel methods. Specifically, we combine two classical algorithms--Nadaraya-Watson (NW) regression or kernel smoothing, and kernel ridge regression (KRR)--with KT to provide a quadratic speed-up in both training and inference times. We show how distribution compression with KT in each setting reduces to constructing an appropriate kernel, and introduce the Kernel-Thinned NW and Kernel-Thinned KRR estimators. We prove that KT-based regression estimators enjoy significantly superior computational efficiency over the full-data estimators and improved statistical efficiency over i.i.d. subsampling of the training data. En route, we also provide a novel multiplicative error guarantee for compressing with KT. We validate our design choices with both simulations and real data experiments.","url_abs":"https://arxiv.org/abs/2410.13749v2","url_pdf":"https://arxiv.org/pdf/2410.13749v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"supervised-kernel-thinning","repo_url":"https://github.com/ag2435/npr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"coresets","method_name":"Coresets"},{"method_slug":"set","method_name":"SET"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2410.13749","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.13749"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ag2435/npr","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"ran":10,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c288f1ca96210258","entry":"euclidean_distances","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_k.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_k.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c288f1ca96210258"}},{"code_sha256_prefix":"28405ebd4b1a43ad","entry":"get_coreset_size","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_thin_dnc.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_thin_dnc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"28405ebd4b1a43ad"}},{"code_sha256_prefix":"48d9057a107a8bd5","entry":"get_g","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_thin.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_thin.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"48d9057a107a8bd5"}},{"code_sha256_prefix":"aeda13fd13f0720b","entry":"grid","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_sample.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"aeda13fd13f0720b"}},{"code_sha256_prefix":"5863f775093fc87c","entry":"log4","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_thin.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_thin.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5863f775093fc87c"}},{"code_sha256_prefix":"7ff5a16c39b40f14","entry":"numpy_to_torch","repo":"ag2435/npr","repo_kind":"official","path":"npr/falkon/util_falkon_estimators.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/falkon/util_falkon_estimators.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"7ff5a16c39b40f14"}},{"code_sha256_prefix":"07a20869feecd04b","entry":"pdf","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_sample.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"07a20869feecd04b"}},{"code_sha256_prefix":"78aec322fa1d2623","entry":"sample","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_sample.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"78aec322fa1d2623"}},{"code_sha256_prefix":"5181cca6881c5fc5","entry":"to_product_kernel","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_k.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_k.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5181cca6881c5fc5"}},{"code_sha256_prefix":"80da1f69b8b758cd","entry":"to_regression_kernel","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_k.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_k.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"80da1f69b8b758cd"}},{"code_sha256_prefix":"dc1bd93d107d4ab8","entry":"add_noise","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_load_data.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_load_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"dc1bd93d107d4ab8"}},{"code_sha256_prefix":"1ddab9f1ba46889a","entry":"get_feature_matrix","repo":"ag2435/npr","repo_kind":"official","path":"npr/krr/utils.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/krr/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1ddab9f1ba46889a"}},{"code_sha256_prefix":"2ed6ff7802b14b5c","entry":"get_svhn_dataset","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_load_data.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_load_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2ed6ff7802b14b5c"}},{"code_sha256_prefix":"c8859ad2976dc5ea","entry":"normalize","repo":"ag2435/npr","repo_kind":"official","path":"npr/util_load_data.py","file_url":"https://github.com/ag2435/npr/blob/HEAD/npr/util_load_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c8859ad2976dc5ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}