{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rest-enhancing-group-robustness-in-dnns","title":"REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training","arxiv_id":"2312.03044","date":"2023-12-05","proceeding":null,"authors":["Jiaxu Zhao","Lu Yin","Shiwei Liu","Meng Fang","Mykola Pechenizkiy"],"abstract":"The deep neural network (DNN) has been proven effective in various domains. However, they often struggle to perform well on certain minority groups during inference, despite showing strong performance on the majority of data groups. This is because over-parameterized models learned \\textit{bias attributes} from a large number of \\textit{bias-aligned} training samples. These bias attributes are strongly spuriously correlated with the target variable, causing the models to be biased towards spurious correlations (i.e., \\textit{bias-conflicting}). To tackle this issue, we propose a novel \\textbf{re}weighted \\textbf{s}parse \\textbf{t}raining framework, dubbed as \\textit{\\textbf{REST}}, which aims to enhance the performance of biased data while improving computation and memory efficiency. Our proposed REST framework has been experimentally validated on three datasets, demonstrating its effectiveness in exploring unbiased subnetworks. We found that REST reduces the reliance on spuriously correlated features, leading to better performance across a wider range of data groups with fewer training and inference resources. We highlight that the \\textit{REST} framework represents a promising approach for improving the performance of DNNs on biased data, while simultaneously improving computation and memory efficiency. By reducing the reliance on spurious correlations, REST has the potential to enhance the robustness of DNNs and improve their generalization capabilities. Code is released at \\url{https://github.com/zhao1402072392/REST}","url_abs":"https://arxiv.org/abs/2312.03044v2","url_pdf":"https://arxiv.org/pdf/2312.03044v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rest-enhancing-group-robustness-in-dnns","repo_url":"https://github.com/zhao1402072392/rest","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.03044","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.03044"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhao1402072392/rest","reach":{"status":"ok"}}],"summary":{"ran":9,"ran_draft_wrong":2,"unverified":6},"by_repo_kind":{"official":{"samples":17,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"0b98222d60088756","entry":"ResNet18","repo":"zhao1402072392/rest","repo_kind":"official","path":"sparselearning/models.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/sparselearning/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b98222d60088756"}},{"code_sha256_prefix":"69f1881e9527c23c","entry":"ResNet34","repo":"zhao1402072392/rest","repo_kind":"official","path":"sparselearning/models.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/sparselearning/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69f1881e9527c23c"}},{"code_sha256_prefix":"a44b24a44c793daa","entry":"ResNet50","repo":"zhao1402072392/rest","repo_kind":"official","path":"sparselearning/models.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/sparselearning/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a44b24a44c793daa"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"zhao1402072392/rest","repo_kind":"official","path":"extractable_resnet.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/extractable_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"zhao1402072392/rest","repo_kind":"official","path":"extractable_resnet.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/extractable_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"1c6dbb2f211c7f04","entry":"count_model_param_flops","repo":"zhao1402072392/rest","repo_kind":"official","path":"flops.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/flops.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c6dbb2f211c7f04"}},{"code_sha256_prefix":"a9c02346cafc6306","entry":"get_acc","repo":"zhao1402072392/rest","repo_kind":"official","path":"chart.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/chart.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9c02346cafc6306"}},{"code_sha256_prefix":"142a262930515741","entry":"get_acc_list","repo":"zhao1402072392/rest","repo_kind":"official","path":"chart.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/chart.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"142a262930515741"}},{"code_sha256_prefix":"9d1aed5c27329c07","entry":"get_projections","repo":"zhao1402072392/rest","repo_kind":"official","path":"random_feature_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/random_feature_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d1aed5c27329c07"}},{"code_sha256_prefix":"6bee5175f6c9e4a2","entry":"get_random_features","repo":"zhao1402072392/rest","repo_kind":"official","path":"random_feature_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/random_feature_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bee5175f6c9e4a2"}},{"code_sha256_prefix":"0a57f8c6ed773a07","entry":"sample_from_sphere","repo":"zhao1402072392/rest","repo_kind":"official","path":"random_feature_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/random_feature_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a57f8c6ed773a07"}},{"code_sha256_prefix":"381380e09fd105f8","entry":"get_accs_for_epoch_across_batches","repo":"zhao1402072392/rest","repo_kind":"official","path":"analysis_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/analysis_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"381380e09fd105f8"}},{"code_sha256_prefix":"38b084886804dc24","entry":"get_all_csv","repo":"zhao1402072392/rest","repo_kind":"official","path":"chart.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/chart.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38b084886804dc24"}},{"code_sha256_prefix":"10e220982fed5f01","entry":"load_log","repo":"zhao1402072392/rest","repo_kind":"official","path":"analysis_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/analysis_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"10e220982fed5f01"}},{"code_sha256_prefix":"bed7fe4dae1cc07a","entry":"resnet10vw","repo":"zhao1402072392/rest","repo_kind":"official","path":"variable_width_resnet.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/variable_width_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bed7fe4dae1cc07a"}},{"code_sha256_prefix":"3c5da610f0879a76","entry":"resnet18","repo":"zhao1402072392/rest","repo_kind":"official","path":"extractable_resnet.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/extractable_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c5da610f0879a76"}},{"code_sha256_prefix":"8a66892616008c7f","entry":"sanitize_df","repo":"zhao1402072392/rest","repo_kind":"official","path":"analysis_utils.py","file_url":"https://github.com/zhao1402072392/rest/blob/HEAD/analysis_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a66892616008c7f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}