{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/separating-common-from-salient-patterns-with","title":"Separating common from salient patterns with Contrastive Representation Learning","arxiv_id":"2402.11928","date":"2024-02-19","proceeding":null,"authors":["Robin Louiset","Edouard Duchesnay","Antoine Grigis","Pietro Gori"],"abstract":"Contrastive Analysis is a sub-field of Representation Learning that aims at separating common factors of variation between two datasets, a background (i.e., healthy subjects) and a target (i.e., diseased subjects), from the salient factors of variation, only present in the target dataset. Despite their relevance, current models based on Variational Auto-Encoders have shown poor performance in learning semantically-expressive representations. On the other hand, Contrastive Representation Learning has shown tremendous performance leaps in various applications (classification, clustering, etc.). In this work, we propose to leverage the ability of Contrastive Learning to learn semantically expressive representations well adapted for Contrastive Analysis. We reformulate it under the lens of the InfoMax Principle and identify two Mutual Information terms to maximize and one to minimize. We decompose the first two terms into an Alignment and a Uniformity term, as commonly done in Contrastive Learning. Then, we motivate a novel Mutual Information minimization strategy to prevent information leakage between common and salient distributions. We validate our method, called SepCLR, on three visual datasets and three medical datasets, specifically conceived to assess the pattern separation capability in Contrastive Analysis. Code available at https://github.com/neurospin-projects/2024_rlouiset_sep_clr.","url_abs":"https://arxiv.org/abs/2402.11928v1","url_pdf":"https://arxiv.org/pdf/2402.11928v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"separating-common-from-salient-patterns-with","repo_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.11928","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.11928"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Linear95/CLUB","reach":null}],"summary":{"ran_honours":2,"ran_fixture":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"ceefbf03ad4b4c65","entry":"get_batch","repo":"neurospin-projects/2024_rlouiset_sep_clr","repo_kind":"official","path":"celeba_accessories/create_dataset.py","file_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr/blob/HEAD/celeba_accessories/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ceefbf03ad4b4c65"}},{"code_sha256_prefix":"dca13cb42b45478e","entry":"get_image","repo":"neurospin-projects/2024_rlouiset_sep_clr","repo_kind":"official","path":"celeba_accessories/create_dataset.py","file_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr/blob/HEAD/celeba_accessories/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dca13cb42b45478e"}},{"code_sha256_prefix":"a366e46f97a9e86a","entry":"images_square_grid","repo":"neurospin-projects/2024_rlouiset_sep_clr","repo_kind":"official","path":"celeba_accessories/create_dataset.py","file_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr/blob/HEAD/celeba_accessories/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a366e46f97a9e86a"}},{"code_sha256_prefix":"fd2bcce4c270c18b","entry":"joint_entropy_loss","repo":"neurospin-projects/2024_rlouiset_sep_clr","repo_kind":"official","path":"losses.py","file_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd2bcce4c270c18b"}},{"code_sha256_prefix":"abb3b39c6e174088","entry":"norm","repo":"neurospin-projects/2024_rlouiset_sep_clr","repo_kind":"official","path":"cifar_mnist/sep_clr_k-jem.py","file_url":"https://github.com/neurospin-projects/2024_rlouiset_sep_clr/blob/HEAD/cifar_mnist/sep_clr_k-jem.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"abb3b39c6e174088"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}