{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-driven-subgroup-identification-for","title":"Data-Driven Subgroup Identification for Linear Regression","arxiv_id":"2305.00195","date":"2023-04-29","proceeding":null,"authors":["Zachary Izzo","Ruishan Liu","James Zou"],"abstract":"Medical studies frequently require to extract the relationship between each covariate and the outcome with statistical confidence measures. To do this, simple parametric models are frequently used (e.g. coefficients of linear regression) but usually fitted on the whole dataset. However, it is common that the covariates may not have a uniform effect over the whole population and thus a unified simple model can miss the heterogeneous signal. For example, a linear model may be able to explain a subset of the data but fail on the rest due to the nonlinearity and heterogeneity in the data. In this paper, we propose DDGroup (data-driven group discovery), a data-driven method to effectively identify subgroups in the data with a uniform linear relationship between the features and the label. DDGroup outputs an interpretable region in which the linear model is expected to hold. It is simple to implement and computationally tractable for use. We show theoretically that, given a large enough sample, DDGroup recovers a region where a single linear model with low variance is well-specified (if one exists), and experiments on real-world medical datasets confirm that it can discover regions where a local linear model has improved performance. Our experiments also show that DDGroup can uncover subgroups with qualitatively different relationships which are missed by simply applying parametric approaches to the whole dataset.","url_abs":"https://arxiv.org/abs/2305.00195v1","url_pdf":"https://arxiv.org/pdf/2305.00195v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-driven-subgroup-identification-for","repo_url":"https://github.com/zleizzo/ddgroup","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.00195","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.00195"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zleizzo/DDGroup","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zleizzo/ddgroup","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":10,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5f8bea0e892b439a","entry":"directed_infty_norm","repo":"zleizzo/ddgroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/ddgroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f8bea0e892b439a"}},{"code_sha256_prefix":"10e004d99c866235","entry":"find_core","repo":"zleizzo/ddgroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/ddgroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10e004d99c866235"}},{"code_sha256_prefix":"94df95305777302f","entry":"find_core","repo":"zleizzo/DDGroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/DDGroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94df95305777302f"}},{"code_sha256_prefix":"0f8e2c19065fc212","entry":"generate_clss_data","repo":"zleizzo/DDGroup","repo_kind":"official","path":"src/utils/data_loader.py","file_url":"https://github.com/zleizzo/DDGroup/blob/HEAD/src/utils/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f8e2c19065fc212"}},{"code_sha256_prefix":"6b537d56e82ea5a5","entry":"generate_regr_data","repo":"zleizzo/DDGroup","repo_kind":"official","path":"src/utils/data_loader.py","file_url":"https://github.com/zleizzo/DDGroup/blob/HEAD/src/utils/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b537d56e82ea5a5"}},{"code_sha256_prefix":"87b2991fe2f63fa6","entry":"hard_grow_region","repo":"zleizzo/ddgroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/ddgroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87b2991fe2f63fa6"}},{"code_sha256_prefix":"dd1ada9d5fa20645","entry":"neighbor_core","repo":"zleizzo/ddgroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/ddgroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd1ada9d5fa20645"}},{"code_sha256_prefix":"063f3568eba03070","entry":"neighbor_core","repo":"zleizzo/DDGroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/DDGroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"063f3568eba03070"}},{"code_sha256_prefix":"f05b8eb58946ed9e","entry":"res","repo":"zleizzo/ddgroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/ddgroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f05b8eb58946ed9e"}},{"code_sha256_prefix":"e81169bbb6fb38ee","entry":"res","repo":"zleizzo/DDGroup","repo_kind":"official","path":"src/utils/methods.py","file_url":"https://github.com/zleizzo/DDGroup/blob/HEAD/src/utils/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e81169bbb6fb38ee"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}