{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/combating-representation-learning-disparity-1","title":"Combating Representation Learning Disparity with Geometric Harmonization","arxiv_id":"2310.17622","date":"2023-10-26","proceeding":"NeurIPS 2023 11","authors":["Zhihan Zhou","Jiangchao Yao","Feng Hong","Ya zhang","Bo Han","Yanfeng Wang"],"abstract":"Self-supervised learning (SSL) as an effective paradigm of representation learning has achieved tremendous success on various curated datasets in diverse scenarios. Nevertheless, when facing the long-tailed distribution in real-world applications, it is still hard for existing methods to capture transferable and robust representation. Conventional SSL methods, pursuing sample-level uniformity, easily leads to representation learning disparity where head classes dominate the feature regime but tail classes passively collapse. To address this problem, we propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning, which is more benign to the minority and almost does not hurt the majority under long-tailed distribution. Specially, GH measures the population statistics of the embedding space on top of self-supervised learning, and then infer an fine-grained instance-wise calibration to constrain the space expansion of head classes and avoid the passive collapse of tail classes. Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner. Extensive results on a range of benchmark datasets show the effectiveness of GH with high tolerance to the distribution skewness. Our code is available at https://github.com/MediaBrain-SJTU/Geometric-Harmonization.","url_abs":"https://arxiv.org/abs/2310.17622v1","url_pdf":"https://arxiv.org/pdf/2310.17622v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"combating-representation-learning-disparity-1","repo_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.17622","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.17622"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_draft_wrong":5,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"43a1d3b25104ae7a","entry":"ResNet18","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"resnet.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43a1d3b25104ae7a"}},{"code_sha256_prefix":"c2b0a60d47724297","entry":"ResNet34","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"resnet.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2b0a60d47724297"}},{"code_sha256_prefix":"9846307204b08173","entry":"ShearX","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"BCL/randaug.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/BCL/randaug.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9846307204b08173"}},{"code_sha256_prefix":"109155544d6d2c48","entry":"ShearY","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"BCL/randaug.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/BCL/randaug.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"109155544d6d2c48"}},{"code_sha256_prefix":"f03d112eee854337","entry":"TranslateX","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"BCL/randaug.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/BCL/randaug.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f03d112eee854337"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"resnet_imagenet.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/resnet_imagenet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"1cd86864f32ad467","entry":"conv1x1","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"SDCLR/resnet_prune.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/SDCLR/resnet_prune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1cd86864f32ad467"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"resnet_imagenet.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/resnet_imagenet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"ae216d6807b93383","entry":"conv3x3","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"SDCLR/resnet_prune.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/SDCLR/resnet_prune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae216d6807b93383"}},{"code_sha256_prefix":"0e0f66ffb7a61530","entry":"normalize_fn","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"resnet.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/resnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e0f66ffb7a61530"}},{"code_sha256_prefix":"38b1cfaaa3fc1906","entry":"sinkhorn","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"utils.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38b1cfaaa3fc1906"}},{"code_sha256_prefix":"21004f703e266a6c","entry":"calculate_momentum_weight","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"BCL/memoboosted_LT_Dataset.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/BCL/memoboosted_LT_Dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21004f703e266a6c"}},{"code_sha256_prefix":"01e541288a411510","entry":"calculate_momentum_weight","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"BCL/memoboosted_cifar100.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/BCL/memoboosted_cifar100.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01e541288a411510"}},{"code_sha256_prefix":"b09a2cc2e21f2012","entry":"get_negative_mask","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"utils.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b09a2cc2e21f2012"}},{"code_sha256_prefix":"1223e88b708e9e08","entry":"nt_xent","repo":"MediaBrain-SJTU/Geometric-Harmonization","repo_kind":"official","path":"utils.py","file_url":"https://github.com/MediaBrain-SJTU/Geometric-Harmonization/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1223e88b708e9e08"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}