{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-the-uniformity-metric-in-self","title":"Rethinking The Uniformity Metric in Self-Supervised Learning","arxiv_id":"2403.00642","date":"2024-03-01","proceeding":null,"authors":["Xianghong Fang","Jian Li","Qiang Sun","Benyou Wang"],"abstract":"Uniformity plays an important role in evaluating learned representations, providing insights into self-supervised learning. In our quest for effective uniformity metrics, we pinpoint four principled properties that such metrics should possess. Namely, an effective uniformity metric should remain invariant to instance permutations and sample replications while accurately capturing feature redundancy and dimensional collapse. Surprisingly, we find that the uniformity metric proposed by \\citet{Wang2020UnderstandingCR} fails to satisfy the majority of these properties. Specifically, their metric is sensitive to sample replications, and can not account for feature redundancy and dimensional collapse correctly. To overcome these limitations, we introduce a new uniformity metric based on the Wasserstein distance, which satisfies all the aforementioned properties. Integrating this new metric in existing self-supervised learning methods effectively mitigates dimensional collapse and consistently improves their performance on downstream tasks involving CIFAR-10 and CIFAR-100 datasets. Code is available at \\url{https://github.com/statsle/WassersteinSSL}.","url_abs":"https://arxiv.org/abs/2403.00642v2","url_pdf":"https://arxiv.org/pdf/2403.00642v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-the-uniformity-metric-in-self","repo_url":"https://github.com/statsle/wassersteinssl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"rethinking-the-uniformity-metric-in-self","repo_url":"https://github.com/sunset-clouds/wassersteinuniformitymetric","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.00642","atlas_url":"https://app.syntology.ai/?focus=2403.00642","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.00642"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunset-clouds/wassersteinuniformitymetric","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/statsle/wassersteinssl","reach":{"status":"ok"}}],"summary":{"ran":5},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"3208dc8dfe2a57e1","entry":"calc_entropy","repo":"statsle/wassersteinssl","repo_kind":"official","path":"DistributionApproximation/metric.py","file_url":"https://github.com/statsle/wassersteinssl/blob/HEAD/DistributionApproximation/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3208dc8dfe2a57e1"}},{"code_sha256_prefix":"7efbbe0ea4dbe265","entry":"calc_wasserstein_distance","repo":"statsle/wassersteinssl","repo_kind":"official","path":"DistributionApproximation/metric.py","file_url":"https://github.com/statsle/wassersteinssl/blob/HEAD/DistributionApproximation/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7efbbe0ea4dbe265"}},{"code_sha256_prefix":"a88c8c7ca37c7ed9","entry":"density_estimate","repo":"statsle/wassersteinssl","repo_kind":"official","path":"DistributionApproximation/Density1DPlot.py","file_url":"https://github.com/statsle/wassersteinssl/blob/HEAD/DistributionApproximation/Density1DPlot.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a88c8c7ca37c7ed9"}},{"code_sha256_prefix":"60affc4f01ee7c6a","entry":"density_estimate","repo":"statsle/wassersteinssl","repo_kind":"official","path":"DistributionApproximation/Density2DPlot.py","file_url":"https://github.com/statsle/wassersteinssl/blob/HEAD/DistributionApproximation/Density2DPlot.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"60affc4f01ee7c6a"}},{"code_sha256_prefix":"8d3d5f756818e024","entry":"uniform_loss","repo":"statsle/wassersteinssl","repo_kind":"official","path":"DistributionApproximation/metric.py","file_url":"https://github.com/statsle/wassersteinssl/blob/HEAD/DistributionApproximation/metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d3d5f756818e024"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}