{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/approximate-cross-validation-for-structured","title":"Approximate Cross-Validation for Structured Models","arxiv_id":"2006.12669","date":"2020-06-23","proceeding":"NeurIPS 2020 12","authors":["Soumya Ghosh","William T. Stephenson","Tin D. Nguyen","Sameer K. Deshpande","Tamara Broderick"],"abstract":"Many modern data analyses benefit from explicitly modeling dependence structure in data -- such as measurements across time or space, ordered words in a sentence, or genes in a genome. A gold standard evaluation technique is structured cross-validation (CV), which leaves out some data subset (such as data within a time interval or data in a geographic region) in each fold. But CV here can be prohibitively slow due to the need to re-run already-expensive learning algorithms many times. Previous work has shown approximate cross-validation (ACV) methods provide a fast and provably accurate alternative in the setting of empirical risk minimization. But this existing ACV work is restricted to simpler models by the assumptions that (i) data across CV folds are independent and (ii) an exact initial model fit is available. In structured data analyses, both these assumptions are often untrue. In the present work, we address (i) by extending ACV to CV schemes with dependence structure between the folds. To address (ii), we verify -- both theoretically and empirically -- that ACV quality deteriorates smoothly with noise in the initial fit. We demonstrate the accuracy and computational benefits of our proposed methods on a diverse set of real-world applications.","url_abs":"https://arxiv.org/abs/2006.12669v2","url_pdf":"https://arxiv.org/pdf/2006.12669v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"approximate-cross-validation-for-structured","repo_url":"https://github.com/SoumyaTGhosh/structured-infinitesimal-jackknife","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.12669","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.12669"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SoumyaTGhosh/structured-infinitesimal-jackknife","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9c218c772ca64f91","entry":"extract_folds","repo":"SoumyaTGhosh/structured-infinitesimal-jackknife","repo_kind":"official","path":"structij/utils.py","file_url":"https://github.com/SoumyaTGhosh/structured-infinitesimal-jackknife/blob/HEAD/structij/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c218c772ca64f91"}},{"code_sha256_prefix":"350c658b3f954e28","entry":"genSyntheticDataset","repo":"SoumyaTGhosh/structured-infinitesimal-jackknife","repo_kind":"official","path":"structij/utils.py","file_url":"https://github.com/SoumyaTGhosh/structured-infinitesimal-jackknife/blob/HEAD/structij/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"350c658b3f954e28"}},{"code_sha256_prefix":"8f1f23227f3d950c","entry":"get_indices_in_held_out_fold","repo":"SoumyaTGhosh/structured-infinitesimal-jackknife","repo_kind":"official","path":"structij/utils.py","file_url":"https://github.com/SoumyaTGhosh/structured-infinitesimal-jackknife/blob/HEAD/structij/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f1f23227f3d950c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}