{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autobalance-optimized-loss-functions-for-1","title":"AutoBalance: Optimized Loss Functions for Imbalanced Data","arxiv_id":"2201.01212","date":"2022-01-04","proceeding":"NeurIPS 2021 12","authors":["Mingchen Li","Xuechen Zhang","Christos Thrampoulidis","Jiasi Chen","Samet Oymak"],"abstract":"Imbalanced datasets are commonplace in modern machine learning problems. The presence of under-represented classes or groups with sensitive attributes results in concerns about generalization and fairness. Such concerns are further exacerbated by the fact that large capacity deep nets can perfectly fit the training data and appear to achieve perfect accuracy and fairness during training, but perform poorly during test. To address these challenges, we propose AutoBalance, a bi-level optimization framework that automatically designs a training loss function to optimize a blend of accuracy and fairness-seeking objectives. Specifically, a lower-level problem trains the model weights, and an upper-level problem tunes the loss function by monitoring and optimizing the desired objective over the validation data. Our loss design enables personalized treatment for classes/groups by employing a parametric cross-entropy loss and individualized data augmentation schemes. We evaluate the benefits and performance of our approach for the application scenarios of imbalanced and group-sensitive classification. Extensive empirical evaluations demonstrate the benefits of AutoBalance over state-of-the-art approaches. Our experimental findings are complemented with theoretical insights on loss function design and the benefits of train-validation split. All code is available open-source.","url_abs":"https://arxiv.org/abs/2201.01212v1","url_pdf":"https://arxiv.org/pdf/2201.01212v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autobalance-optimized-loss-functions-for-1","repo_url":"https://github.com/ucr-optml/autobalance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2201.01212","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.01212"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ucr-optml/autobalance","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_fixture":2,"ran_honours":3,"unverified":8},"by_repo_kind":{"official":{"samples":14,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ac269a0e4b8d946e","entry":"default_loader","repo":"ucr-optml/autobalance","repo_kind":"official","path":"dataset/iNaturalist.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/dataset/iNaturalist.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac269a0e4b8d946e"}},{"code_sha256_prefix":"2dd5dfa94542911e","entry":"drop_connect","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2dd5dfa94542911e"}},{"code_sha256_prefix":"739afdca8b2d673a","entry":"gather_flat_grad","repo":"ucr-optml/autobalance","repo_kind":"official","path":"core/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"739afdca8b2d673a"}},{"code_sha256_prefix":"085767456c708d9d","entry":"neumann_hyperstep_preconditioner","repo":"ucr-optml/autobalance","repo_kind":"official","path":"core/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"085767456c708d9d"}},{"code_sha256_prefix":"826e2e91d6a9ec09","entry":"round_filters","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"826e2e91d6a9ec09"}},{"code_sha256_prefix":"4d8e794b593d34a0","entry":"round_repeats","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d8e794b593d34a0"}},{"code_sha256_prefix":"a299832f4fcf0211","entry":"ResNet20","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a299832f4fcf0211"}},{"code_sha256_prefix":"80b1b62fc8826034","entry":"ResNet32","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80b1b62fc8826034"}},{"code_sha256_prefix":"ec1ce8e626c3703d","entry":"ResNet44","repo":"ucr-optml/autobalance","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec1ce8e626c3703d"}},{"code_sha256_prefix":"41d10ece707b44af","entry":"eval_epoch","repo":"ucr-optml/autobalance","repo_kind":"official","path":"core/trainer.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/core/trainer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41d10ece707b44af"}},{"code_sha256_prefix":"8bf3be98e6cb32cb","entry":"load_cifar10","repo":"ucr-optml/autobalance","repo_kind":"official","path":"dataset/cifar10.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/dataset/cifar10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bf3be98e6cb32cb"}},{"code_sha256_prefix":"297d16d00bf739bb","entry":"load_cifar100","repo":"ucr-optml/autobalance","repo_kind":"official","path":"dataset/cifar100.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/dataset/cifar100.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"297d16d00bf739bb"}},{"code_sha256_prefix":"cac75c3f585e4df2","entry":"load_taxonomy","repo":"ucr-optml/autobalance","repo_kind":"official","path":"dataset/iNaturalist.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/dataset/iNaturalist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cac75c3f585e4df2"}},{"code_sha256_prefix":"1025abdbfd524442","entry":"loss_adjust_cross_entropy","repo":"ucr-optml/autobalance","repo_kind":"official","path":"core/utils.py","file_url":"https://github.com/ucr-optml/autobalance/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1025abdbfd524442"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}