{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bridging-mini-batch-and-asymptotic-analysis","title":"Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses","arxiv_id":"2405.18045","date":"2024-05-28","proceeding":null,"authors":["Panagiotis Koromilas","Giorgos Bouritsas","Theodoros Giannakopoulos","Mihalis Nicolaou","Yannis Panagakis"],"abstract":"What do different contrastive learning (CL) losses actually optimize for? Although multiple CL methods have demonstrated remarkable representation learning capabilities, the differences in their inner workings remain largely opaque. In this work, we analyse several CL families and prove that, under certain conditions, they admit the same minimisers when optimizing either their batch-level objectives or their expectations asymptotically. In both cases, an intimate connection with the hyperspherical energy minimisation (HEM) problem resurfaces. Drawing inspiration from this, we introduce a novel CL objective, coined Decoupled Hyperspherical Energy Loss (DHEL). DHEL simplifies the problem by decoupling the target hyperspherical energy from the alignment of positive examples while preserving the same theoretical guarantees. Going one step further, we show the same results hold for another relevant CL family, namely kernel contrastive learning (KCL), with the additional advantage of the expected loss being independent of batch size, thus identifying the minimisers in the non-asymptotic regime. Empirical results demonstrate improved downstream performance and robustness across combinations of different batch sizes and hyperparameters and reduced dimensionality collapse, on several computer vision datasets.","url_abs":"https://arxiv.org/abs/2405.18045v1","url_pdf":"https://arxiv.org/pdf/2405.18045v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bridging-mini-batch-and-asymptotic-analysis","repo_url":"https://github.com/pakoromilas/dhel-kcl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"dhel","method_name":"DHEL"}],"datasets_introduced":[],"methods_introduced":[{"slug":"dhel","name":"DHEL","full_name":"Decoupled Hyperspherical Energy Loss"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.18045","atlas_url":"https://app.syntology.ai/?focus=2405.18045","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18045"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/pakoromilas/DHEL-KCL","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pakoromilas/dhel-kcl","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"d9279963eb3c63d0","entry":"DHEL","repo":"pakoromilas/dhel-kcl","repo_kind":"official","path":"losses.py","file_url":"https://github.com/pakoromilas/dhel-kcl/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9279963eb3c63d0"}},{"code_sha256_prefix":"03a3d9fa300ca205","entry":"align_gaussian","repo":"pakoromilas/DHEL-KCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/pakoromilas/DHEL-KCL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03a3d9fa300ca205"}},{"code_sha256_prefix":"ffce905de6898516","entry":"alignment","repo":"pakoromilas/dhel-kcl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/pakoromilas/dhel-kcl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffce905de6898516"}},{"code_sha256_prefix":"3b6a60a016c59031","entry":"alignment","repo":"pakoromilas/DHEL-KCL","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/pakoromilas/DHEL-KCL/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b6a60a016c59031"}},{"code_sha256_prefix":"01e2c8ff869dd49d","entry":"gaussian_kernel","repo":"pakoromilas/DHEL-KCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/pakoromilas/DHEL-KCL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01e2c8ff869dd49d"}},{"code_sha256_prefix":"da0f787095556e25","entry":"riesz_kernel","repo":"pakoromilas/DHEL-KCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/pakoromilas/DHEL-KCL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da0f787095556e25"}},{"code_sha256_prefix":"4f764f2a19f0b77c","entry":"uniformity","repo":"pakoromilas/dhel-kcl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/pakoromilas/dhel-kcl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f764f2a19f0b77c"}},{"code_sha256_prefix":"a76734d3c3093d0d","entry":"uniformity","repo":"pakoromilas/DHEL-KCL","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/pakoromilas/DHEL-KCL/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a76734d3c3093d0d"}},{"code_sha256_prefix":"4ef221abdc61ae51","entry":"wasserstein_uniformity","repo":"pakoromilas/dhel-kcl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/pakoromilas/dhel-kcl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ef221abdc61ae51"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}