{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-nonparametric-statistical-inference","title":"Efficient nonparametric statistical inference on population feature importance using Shapley values","arxiv_id":"2006.09481","date":"2020-06-16","proceeding":"ICML 2020 1","authors":["Brian D. Williamson","Jean Feng"],"abstract":"The true population-level importance of a variable in a prediction task provides useful knowledge about the underlying data-generating mechanism and can help in deciding which measurements to collect in subsequent experiments. Valid statistical inference on this importance is a key component in understanding the population of interest. We present a computationally efficient procedure for estimating and obtaining valid statistical inference on the Shapley Population Variable Importance Measure (SPVIM). Although the computational complexity of the true SPVIM scales exponentially with the number of variables, we propose an estimator based on randomly sampling only $\\Theta(n)$ feature subsets given $n$ observations. We prove that our estimator converges at an asymptotically optimal rate. Moreover, by deriving the asymptotic distribution of our estimator, we construct valid confidence intervals and hypothesis tests. Our procedure has good finite-sample performance in simulations, and for an in-hospital mortality prediction task produces similar variable importance estimates when different machine learning algorithms are applied.","url_abs":"https://arxiv.org/abs/2006.09481v1","url_pdf":"https://arxiv.org/pdf/2006.09481v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-nonparametric-statistical-inference","repo_url":"https://github.com/bdwilliamson/spvim_supplementary","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"efficient-nonparametric-statistical-inference","repo_url":"https://github.com/bdwilliamson/vimp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"efficient-nonparametric-statistical-inference","repo_url":"https://github.com/bdwilliamson/vimpy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"mortality-prediction","task_name":"Mortality Prediction"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.09481","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.09481"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bdwilliamson/spvim_supplementary","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bdwilliamson/vimp","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bdwilliamson/vimpy","reach":null}],"summary":{"ran_fixture":5,"ran":8,"ran_draft_wrong":1,"ran_honours":1,"unverified":10},"by_repo_kind":{"listed":{"samples":25,"ran":15,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5efa7eb653ea329a","entry":"accuracy","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5efa7eb653ea329a"}},{"code_sha256_prefix":"b3b4f79c3ab9cdf9","entry":"auc","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3b4f79c3ab9cdf9"}},{"code_sha256_prefix":"514d20b055ecbe1b","entry":"auc_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"514d20b055ecbe1b"}},{"code_sha256_prefix":"96c2d58d78728d79","entry":"choose","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96c2d58d78728d79"}},{"code_sha256_prefix":"64ed85c06ce35e76","entry":"cross_entropy","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64ed85c06ce35e76"}},{"code_sha256_prefix":"f22751c60cbae69a","entry":"deviance","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f22751c60cbae69a"}},{"code_sha256_prefix":"a6fd8ce09cf07e2f","entry":"make_folds","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6fd8ce09cf07e2f"}},{"code_sha256_prefix":"315e68efb0762ec6","entry":"one_auc_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"315e68efb0762ec6"}},{"code_sha256_prefix":"38bb504d66f788f8","entry":"one_r2_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38bb504d66f788f8"}},{"code_sha256_prefix":"534144f19cbd0ac6","entry":"r_squared","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"534144f19cbd0ac6"}},{"code_sha256_prefix":"85ce6da752e9ae14","entry":"r_squared_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85ce6da752e9ae14"}},{"code_sha256_prefix":"8e32ea0b9aa76ef2","entry":"shapley_hyp_test","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e32ea0b9aa76ef2"}},{"code_sha256_prefix":"610ae143866b4724","entry":"shapley_influence_function","repo":"bdwilliamson/spvim_supplementary","repo_kind":"listed","path":"code/data_analysis/get_influence_functions.py","file_url":"https://github.com/bdwilliamson/spvim_supplementary/blob/HEAD/code/data_analysis/get_influence_functions.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"610ae143866b4724"}},{"code_sha256_prefix":"8468321fa0661658","entry":"shapley_influence_function","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8468321fa0661658"}},{"code_sha256_prefix":"98c482a48f51a3d5","entry":"shapley_se","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98c482a48f51a3d5"}},{"code_sha256_prefix":"af13607054544c80","entry":"accuracy_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af13607054544c80"}},{"code_sha256_prefix":"559611a6ba26946d","entry":"compute_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"559611a6ba26946d"}},{"code_sha256_prefix":"f40e6267444d9487","entry":"cross_entropy_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f40e6267444d9487"}},{"code_sha256_prefix":"1c5b419e1c08be46","entry":"cv_predictiveness","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c5b419e1c08be46"}},{"code_sha256_prefix":"b03f39be86306d13","entry":"deviance_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b03f39be86306d13"}},{"code_sha256_prefix":"7834085f6f2c900c","entry":"get_measure_function","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7834085f6f2c900c"}},{"code_sha256_prefix":"6e65685b71563fac","entry":"one_accuracy_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e65685b71563fac"}},{"code_sha256_prefix":"2d1e9e7840f75757","entry":"one_cross_entropy_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d1e9e7840f75757"}},{"code_sha256_prefix":"434a920696b14b3d","entry":"one_deviance_ic","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"434a920696b14b3d"}},{"code_sha256_prefix":"f65b08b4141edc84","entry":"spvim","repo":"bdwilliamson/vimpy","repo_kind":"listed","path":"vimpy/spvim.py","file_url":"https://github.com/bdwilliamson/vimpy/blob/HEAD/vimpy/spvim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f65b08b4141edc84"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}