{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/propensity-score-alignment-of-unpaired","title":"Propensity Score Alignment of Unpaired Multimodal Data","arxiv_id":"2404.01595","date":"2024-04-02","proceeding":null,"authors":["Johnny Xi","Jana Osea","Zuheng Xu","Jason Hartford"],"abstract":"Multimodal representation learning techniques typically rely on paired samples to learn common representations, but paired samples are challenging to collect in fields such as biology where measurement devices often destroy the samples. This paper presents an approach to address the challenge of aligning unpaired samples across disparate modalities in multimodal representation learning. We draw an analogy between potential outcomes in causal inference and potential views in multimodal observations, which allows us to use Rubin's framework to estimate a common space in which to match samples. Our approach assumes we collect samples that are experimentally perturbed by treatments, and uses this to estimate a propensity score from each modality, which encapsulates all shared information between a latent state and treatment and can be used to define a distance between samples. We experiment with two alignment techniques that leverage this distance -- shared nearest neighbours (SNN) and optimal transport (OT) matching -- and find that OT matching results in significant improvements over state-of-the-art alignment approaches in both a synthetic multi-modal setting and in real-world data from NeurIPS Multimodal Single-Cell Integration Challenge.","url_abs":"https://arxiv.org/abs/2404.01595v2","url_pdf":"https://arxiv.org/pdf/2404.01595v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"propensity-score-alignment-of-unpaired","repo_url":"https://github.com/valence-labs/prop-score-pairing","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"causal-inference","task_name":"Causal Inference"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"causal-inference","method_name":"Causal inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.01595","atlas_url":"https://app.syntology.ai/?focus=2404.01595","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.01595"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/valence-labs/prop-score-pairing","reach":null}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"5273f84dec754093","entry":"CoupledDataset","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5273f84dec754093"}},{"code_sha256_prefix":"4552a6338c3978df","entry":"IV_MSELoss","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4552a6338c3978df"}},{"code_sha256_prefix":"0def398bc2c0bd21","entry":"MSEFunction","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0def398bc2c0bd21"}},{"code_sha256_prefix":"66207497a3ef4a30","entry":"MatchingProbe","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66207497a3ef4a30"}},{"code_sha256_prefix":"77e162f5b36400df","entry":"WrapperDataset","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77e162f5b36400df"}},{"code_sha256_prefix":"45b3139a7ee3ceda","entry":"eot_matching","repo":"valence-labs/prop-score-pairing","repo_kind":"official","path":"matching/models/probe.py","file_url":"https://github.com/valence-labs/prop-score-pairing/blob/HEAD/matching/models/probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45b3139a7ee3ceda"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}