{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/panning-for-gold-model-x-knockoffs-for-high","title":"Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection","arxiv_id":"1610.02351","date":"2016-10-07","proceeding":null,"authors":["Emmanuel Candes","Yingying Fan","Lucas Janson","Jinchi Lv"],"abstract":"Many contemporary large-scale applications involve building interpretable models linking a large set of potential covariates to a response in a nonlinear fashion, such as when the response is binary. Although this modeling problem has been extensively studied, it remains unclear how to effectively control the fraction of false discoveries even in high-dimensional logistic regression, not to mention general high-dimensional nonlinear models. To address such a practical problem, we propose a new framework of $model$-$X$ knockoffs, which reads from a different perspective the knockoff procedure (Barber and Cand\\`es, 2015) originally designed for controlling the false discovery rate in linear models. Whereas the knockoffs procedure is constrained to homoscedastic linear models with $n\\ge p$, the key innovation here is that model-X knockoffs provide valid inference from finite samples in settings in which the conditional distribution of the response is arbitrary and completely unknown. Furthermore, this holds no matter the number of covariates. Correct inference in such a broad setting is achieved by constructing knockoff variables probabilistically instead of geometrically. To do this, our approach requires the covariates be random (independent and identically distributed rows) with a distribution that is known, although we provide preliminary experimental evidence that our procedure is robust to unknown/estimated distributions. To our knowledge, no other procedure solves the $controlled$ variable selection problem in such generality, but in the restricted settings where competitors exist, we demonstrate the superior power of knockoffs through simulations. Finally, we apply our procedure to data from a case-control study of Crohn's disease in the United Kingdom, making twice as many discoveries as the original analysis of the same data.","url_abs":"https://arxiv.org/abs/1610.02351v4","url_pdf":"https://arxiv.org/pdf/1610.02351v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"panning-for-gold-model-x-knockoffs-for-high","repo_url":"https://github.com/amspector100/knockpy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"panning-for-gold-model-x-knockoffs-for-high","repo_url":"https://github.com/rajesh-lab/ddlk","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"AGPL-3.0"}},{"paper_slug":"panning-for-gold-model-x-knockoffs-for-high","repo_url":"https://github.com/toamto94/Knockoff-Mixed-Graphical-Model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1610.02351","atlas_url":"https://app.syntology.ai/?focus=1610.02351","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1610.02351"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rajesh-lab/ddlk","reach":{"status":"ok","spdx":"AGPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amspector100/knockpy","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/toamto94/Knockoff-Mixed-Graphical-Model","reach":{"status":"ok"}}],"summary":{"unverified":24},"by_repo_kind":{"listed":{"samples":24,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"da14134814dc0e78","entry":"DirichletCorr","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/dgp.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/dgp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da14134814dc0e78"}},{"code_sha256_prefix":"90012e7ae0e81f67","entry":"UniformDot","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/dgp.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/dgp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90012e7ae0e81f67"}},{"code_sha256_prefix":"d72098be902237e9","entry":"Wishart","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/dgp.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/dgp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72098be902237e9"}},{"code_sha256_prefix":"adc23ee77dec077a","entry":"calc_group_sizes","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/utilities.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/utilities.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"adc23ee77dec077a"}},{"code_sha256_prefix":"aefd4b54e9f03113","entry":"calc_mse","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/knockoff_stats.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/knockoff_stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aefd4b54e9f03113"}},{"code_sha256_prefix":"5458ec42b388ef4f","entry":"cholupdate","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/mrc.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5458ec42b388ef4f"}},{"code_sha256_prefix":"57e1e9e9cc014bfb","entry":"compute_ggm_threshold","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/ggm.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/ggm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57e1e9e9cc014bfb"}},{"code_sha256_prefix":"746a83193745ebea","entry":"create_batches","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/kpytorch/deeppink.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/kpytorch/deeppink.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"746a83193745ebea"}},{"code_sha256_prefix":"420aa22ddd8ec2dd","entry":"discovered_edges","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/ggm.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/ggm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"420aa22ddd8ec2dd"}},{"code_sha256_prefix":"59022b18d4ea7f9c","entry":"divide_computation","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/smatrix.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/smatrix.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59022b18d4ea7f9c"}},{"code_sha256_prefix":"90ca0e3347784798","entry":"fetch_group_nonnulls","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/utilities.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/utilities.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90ca0e3347784798"}},{"code_sha256_prefix":"ce1e7a9ba67294a2","entry":"gaussian_log_likelihood","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/metro.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/metro.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce1e7a9ba67294a2"}},{"code_sha256_prefix":"7cea7af4558ec0ff","entry":"get_ordering","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/metro.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/metro.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7cea7af4558ec0ff"}},{"code_sha256_prefix":"b135d8a164d09c72","entry":"merge_groups","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/smatrix.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/smatrix.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b135d8a164d09c72"}},{"code_sha256_prefix":"ec3479c58d959f97","entry":"mvr_loss","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/mrc.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec3479c58d959f97"}},{"code_sha256_prefix":"9d353e743b90dcec","entry":"parse_logistic_flag","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/knockoff_stats.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/knockoff_stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d353e743b90dcec"}},{"code_sha256_prefix":"f25da86a45abedd6","entry":"parse_method","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/smatrix.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/smatrix.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f25da86a45abedd6"}},{"code_sha256_prefix":"96949d6294410a08","entry":"parse_y_dist","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/knockoff_stats.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/knockoff_stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96949d6294410a08"}},{"code_sha256_prefix":"89a86b326fa4a15d","entry":"preprocess_groups","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/utilities.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/utilities.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89a86b326fa4a15d"}},{"code_sha256_prefix":"be4d3e6ef956e7e3","entry":"produce_FX_knockoffs","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/knockoffs.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/knockoffs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be4d3e6ef956e7e3"}},{"code_sha256_prefix":"69bdab92b1aba81b","entry":"produce_MX_gaussian_knockoffs","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/knockoffs.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/knockoffs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69bdab92b1aba81b"}},{"code_sha256_prefix":"9069f8af21c2d20d","entry":"solve_mvr_factored","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/mrc.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9069f8af21c2d20d"}},{"code_sha256_prefix":"4a5aafb88e81d20d","entry":"t_log_likelihood","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/metro.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/metro.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a5aafb88e81d20d"}},{"code_sha256_prefix":"a0f72bcdefa8751d","entry":"train_deeppink","repo":"amspector100/knockpy","repo_kind":"listed","path":"knockpy/kpytorch/deeppink.py","file_url":"https://github.com/amspector100/knockpy/blob/HEAD/knockpy/kpytorch/deeppink.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0f72bcdefa8751d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}