{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-generated-data-always-help-contrastive","title":"Do Generated Data Always Help Contrastive Learning?","arxiv_id":"2403.12448","date":"2024-03-19","proceeding":null,"authors":["Yifei Wang","Jizhe Zhang","Yisen Wang"],"abstract":"Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augmentations. With the rise of generative models, especially diffusion models, the ability to generate realistic images close to the real data distribution has been well recognized. These generated high-equality images have been successfully applied to enhance contrastive representation learning, a technique termed ``data inflation''. However, we find that the generated data (even from a good diffusion model like DDPM) may sometimes even harm contrastive learning. We investigate the causes behind this failure from the perspective of both data inflation and data augmentation. For the first time, we reveal the complementary roles that stronger data inflation should be accompanied by weaker augmentations, and vice versa. We also provide rigorous theoretical explanations for these phenomena via deriving its generalization bounds under data inflation. Drawing from these insights, we propose Adaptive Inflation (AdaInf), a purely data-centric strategy without introducing any extra computation cost. On benchmark datasets, AdaInf can bring significant improvements for various contrastive learning methods. Notably, without using external data, AdaInf obtains 94.70% linear accuracy on CIFAR-10 with SimCLR, setting a new record that surpasses many sophisticated methods. Code is available at https://github.com/PKU-ML/adainf.","url_abs":"https://arxiv.org/abs/2403.12448v1","url_pdf":"https://arxiv.org/pdf/2403.12448v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-generated-data-always-help-contrastive","repo_url":"https://github.com/pku-ml/adainf","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"generalization-bounds","task_name":"Generalization Bounds"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"colorjitter","method_name":"ColorJitter"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"nt-xent","method_name":"NT-Xent"},{"method_slug":"random-gaussian-blur","method_name":"Random Gaussian Blur"},{"method_slug":"random-resized-crop","method_name":"Random Resized Crop"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"simclr","method_name":"SimCLR"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.12448","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.12448"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/PKU-ML/adainf","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pku-ml/adainf","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"132a949a804eb81e","entry":"barlow_loss_func","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/barlow.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/barlow.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"132a949a804eb81e"}},{"code_sha256_prefix":"273532e06cb3967e","entry":"byol_loss_func","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/byol.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/byol.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"273532e06cb3967e"}},{"code_sha256_prefix":"8916e0f98b777300","entry":"extract_features","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/main_knn.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/main_knn.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8916e0f98b777300"}},{"code_sha256_prefix":"a07d9a7525702b80","entry":"mae_loss_func","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/mae.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/mae.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a07d9a7525702b80"}},{"code_sha256_prefix":"4dd7c3cff0150cb9","entry":"mocov2plus_loss_func","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/mocov2plus.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/mocov2plus.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4dd7c3cff0150cb9"}},{"code_sha256_prefix":"3e6a557bf1ad83d9","entry":"patchify","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/mae.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/mae.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e6a557bf1ad83d9"}},{"code_sha256_prefix":"ce432f2e9653c7a3","entry":"deepclusterv2_loss_func","repo":"PKU-ML/adainf","repo_kind":"official","path":"solo-learn/solo/losses/deepclusterv2.py","file_url":"https://github.com/PKU-ML/adainf/blob/HEAD/solo-learn/solo/losses/deepclusterv2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce432f2e9653c7a3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}