{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pairwise-supervised-contrastive-learning-of","title":"Pairwise Supervised Contrastive Learning of Sentence Representations","arxiv_id":"2109.05424","date":"2021-09-12","proceeding":"EMNLP 2021 11","authors":["Dejiao Zhang","Shang-Wen Li","Wei Xiao","Henghui Zhu","Ramesh Nallapati","Andrew O. Arnold","Bing Xiang"],"abstract":"Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss. Nevertheless, they share a common weakness: sentences in a contradiction pair are not necessarily from different semantic categories. Therefore, optimizing the semantic entailment and contradiction reasoning objective alone is inadequate to capture the high-level semantic structure. The drawback is compounded by the fact that the vanilla siamese or triplet losses only learn from individual sentence pairs or triplets, which often suffer from bad local optima. In this paper, we propose PairSupCon, an instance discrimination based approach aiming to bridge semantic entailment and contradiction understanding with high-level categorical concept encoding. We evaluate PairSupCon on various downstream tasks that involve understanding sentence semantics at different granularities. We outperform the previous state-of-the-art method with $10\\%$--$13\\%$ averaged improvement on eight clustering tasks, and $5\\%$--$6\\%$ averaged improvement on seven semantic textual similarity (STS) tasks.","url_abs":"https://arxiv.org/abs/2109.05424v2","url_pdf":"https://arxiv.org/pdf/2109.05424v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pairwise-supervised-contrastive-learning-of","repo_url":"https://github.com/amazon-research/sentence-representations","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"sts","task_name":"STS"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":null,"task_name":"Triplet"}],"methods":[{"method_slug":"triplet-loss","method_name":"Triplet Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.05424","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.05424"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amazon-research/sentence-representations","reach":null}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6fa6e6f1318fd800","entry":"PairSupConBert","repo":"amazon-research/sentence-representations","repo_kind":"official","path":"PairSupCon/models/Transformers.py","file_url":"https://github.com/amazon-research/sentence-representations/blob/HEAD/PairSupCon/models/Transformers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6fa6e6f1318fd800"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}