{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-scale-kernel-methods-for-independence","title":"Large-Scale Kernel Methods for Independence Testing","arxiv_id":"1606.07892","date":"2016-06-25","proceeding":null,"authors":["Qinyi Zhang","Sarah Filippi","Arthur Gretton","Dino Sejdinovic"],"abstract":"Representations of probability measures in reproducing kernel Hilbert spaces\nprovide a flexible framework for fully nonparametric hypothesis tests of\nindependence, which can capture any type of departure from independence,\nincluding nonlinear associations and multivariate interactions. However, these\napproaches come with an at least quadratic computational cost in the number of\nobservations, which can be prohibitive in many applications. Arguably, it is\nexactly in such large-scale datasets that capturing any type of dependence is\nof interest, so striking a favourable tradeoff between computational efficiency\nand test performance for kernel independence tests would have a direct impact\non their applicability in practice. In this contribution, we provide an\nextensive study of the use of large-scale kernel approximations in the context\nof independence testing, contrasting block-based, Nystrom and random Fourier\nfeature approaches. Through a variety of synthetic data experiments, it is\ndemonstrated that our novel large scale methods give comparable performance\nwith existing methods whilst using significantly less computation time and\nmemory.","url_abs":"http://arxiv.org/abs/1606.07892v1","url_pdf":"http://arxiv.org/pdf/1606.07892v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-scale-kernel-methods-for-independence","repo_url":"https://github.com/IBM/SIC","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1606.07892","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1606.07892"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/SIC","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":17},"by_repo_kind":{"listed":{"samples":17,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d81b7960091f43e3","entry":"compute_fdr","repo":"IBM/SIC","repo_kind":"listed","path":"stattests.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/stattests.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d81b7960091f43e3"}},{"code_sha256_prefix":"6a9bfdd55c282718","entry":"compute_mse","repo":"IBM/SIC","repo_kind":"listed","path":"SIC_imports.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/SIC_imports.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a9bfdd55c282718"}},{"code_sha256_prefix":"d07e6ab7af15ef76","entry":"compute_objective_terms","repo":"IBM/SIC","repo_kind":"listed","path":"SIC_imports.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/SIC_imports.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d07e6ab7af15ef76"}},{"code_sha256_prefix":"d204ab0c6f9c28e9","entry":"describe","repo":"IBM/SIC","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d204ab0c6f9c28e9"}},{"code_sha256_prefix":"014bc653675795b0","entry":"dlist","repo":"IBM/SIC","repo_kind":"listed","path":"plot_results.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/plot_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"014bc653675795b0"}},{"code_sha256_prefix":"ea4d799e3e40806a","entry":"get_data","repo":"IBM/SIC","repo_kind":"listed","path":"plot_results.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/plot_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea4d799e3e40806a"}},{"code_sha256_prefix":"290087b87df2ac2e","entry":"get_devices","repo":"IBM/SIC","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"290087b87df2ac2e"}},{"code_sha256_prefix":"fb54dc11aed4cbf8","entry":"init_D","repo":"IBM/SIC","repo_kind":"listed","path":"modules/models.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/modules/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb54dc11aed4cbf8"}},{"code_sha256_prefix":"74d8b31b37ed27e1","entry":"init_optimizerD","repo":"IBM/SIC","repo_kind":"listed","path":"modules/models.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/modules/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"74d8b31b37ed27e1"}},{"code_sha256_prefix":"cedc42f38350b7b6","entry":"label_axis","repo":"IBM/SIC","repo_kind":"listed","path":"plot_results.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/plot_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cedc42f38350b7b6"}},{"code_sha256_prefix":"8e090872f4c05874","entry":"linreg_reconstruct","repo":"IBM/SIC","repo_kind":"listed","path":"modules/linregression.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/modules/linregression.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e090872f4c05874"}},{"code_sha256_prefix":"0b8190dcac4fd585","entry":"make_data_parallel","repo":"IBM/SIC","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0b8190dcac4fd585"}},{"code_sha256_prefix":"2f5a64db1f7ac832","entry":"minibatch","repo":"IBM/SIC","repo_kind":"listed","path":"SIC_imports.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/SIC_imports.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f5a64db1f7ac832"}},{"code_sha256_prefix":"6c42a9c801164386","entry":"risk_model_fn","repo":"IBM/SIC","repo_kind":"listed","path":"run_baselines.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/run_baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c42a9c801164386"}},{"code_sha256_prefix":"a919d57070b61232","entry":"test_generator","repo":"IBM/SIC","repo_kind":"listed","path":"modules/generators.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/modules/generators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a919d57070b61232"}},{"code_sha256_prefix":"adbde4eae22a3f8f","entry":"train_generator","repo":"IBM/SIC","repo_kind":"listed","path":"modules/generators.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/modules/generators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"adbde4eae22a3f8f"}},{"code_sha256_prefix":"8638295ef8c7c717","entry":"train_sklearn_model","repo":"IBM/SIC","repo_kind":"listed","path":"run_baselines.py","file_url":"https://github.com/IBM/SIC/blob/HEAD/run_baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8638295ef8c7c717"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}