{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrastive-learning-with-hard-negative-1","title":"Contrastive Learning with Hard Negative Samples","arxiv_id":"2010.04592","date":"2020-10-09","proceeding":"ICLR 2021 1","authors":["Joshua Robinson","Ching-Yao Chuang","Suvrit Sra","Stefanie Jegelka"],"abstract":"How can you sample good negative examples for contrastive learning? We argue that, as with metric learning, contrastive learning of representations benefits from hard negative samples (i.e., points that are difficult to distinguish from an anchor point). The key challenge toward using hard negatives is that contrastive methods must remain unsupervised, making it infeasible to adopt existing negative sampling strategies that use true similarity information. In response, we develop a new family of unsupervised sampling methods for selecting hard negative samples where the user can control the hardness. A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible. The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.","url_abs":"https://arxiv.org/abs/2010.04592v2","url_pdf":"https://arxiv.org/pdf/2010.04592v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrastive-learning-with-hard-negative-1","repo_url":"https://github.com/joshr17/HCL","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"metric-learning","task_name":"Metric Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.04592","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.04592"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/joshr17/HCL","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/lajanugen/S2V","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":2,"unverified":6},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1},"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de8796ee0c167c5a","entry":"adj_loss_","repo":"joshr17/hcl","repo_kind":"official","path":"graph/losses.py","file_url":"https://github.com/joshr17/hcl/blob/HEAD/graph/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de8796ee0c167c5a"}},{"code_sha256_prefix":"be77056dd1897169","entry":"get_negative_expectation","repo":"joshr17/HCL","repo_kind":"official","path":"graph/cortex_DIM/functions/gan_losses.py","file_url":"https://github.com/joshr17/HCL/blob/HEAD/graph/cortex_DIM/functions/gan_losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be77056dd1897169"}},{"code_sha256_prefix":"c03feba6bc4986f6","entry":"log_sum_exp","repo":"joshr17/HCL","repo_kind":"official","path":"graph/cortex_DIM/functions/gan_losses.py","file_url":"https://github.com/joshr17/HCL/blob/HEAD/graph/cortex_DIM/functions/gan_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c03feba6bc4986f6"}},{"code_sha256_prefix":"796cd317f00fce83","entry":"parse_example_batch","repo":"lajanugen/S2V","repo_kind":"found_in_text","path":"src/ops/input_ops.py","file_url":"https://github.com/lajanugen/S2V/blob/HEAD/src/ops/input_ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"796cd317f00fce83"}},{"code_sha256_prefix":"bf0cc947768c5673","entry":"prefetch_input_data","repo":"lajanugen/S2V","repo_kind":"found_in_text","path":"src/ops/input_ops.py","file_url":"https://github.com/lajanugen/S2V/blob/HEAD/src/ops/input_ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf0cc947768c5673"}},{"code_sha256_prefix":"b05ca027e5d64bc6","entry":"raise_measure_error","repo":"joshr17/HCL","repo_kind":"official","path":"graph/cortex_DIM/functions/gan_losses.py","file_url":"https://github.com/joshr17/HCL/blob/HEAD/graph/cortex_DIM/functions/gan_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b05ca027e5d64bc6"}},{"code_sha256_prefix":"3b50276530bcaf11","entry":"read_vocab","repo":"lajanugen/S2V","repo_kind":"found_in_text","path":"src/s2v_model.py","file_url":"https://github.com/lajanugen/S2V/blob/HEAD/src/s2v_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b50276530bcaf11"}},{"code_sha256_prefix":"29fde95b295e9702","entry":"read_vocab_embs","repo":"lajanugen/S2V","repo_kind":"found_in_text","path":"src/s2v_model.py","file_url":"https://github.com/lajanugen/S2V/blob/HEAD/src/s2v_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"29fde95b295e9702"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}