{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusion-earth-mover-s-distance-and","title":"Diffusion Earth Mover's Distance and Distribution Embeddings","arxiv_id":"2102.12833","date":"2021-02-25","proceeding":null,"authors":["Alexander Tong","Guillaume Huguet","Amine Natik","Kincaid MacDonald","Manik Kuchroo","Ronald Coifman","Guy Wolf","Smita Krishnaswamy"],"abstract":"We propose a new fast method of measuring distances between large numbers of related high dimensional datasets called the Diffusion Earth Mover's Distance (EMD). We model the datasets as distributions supported on common data graph that is derived from the affinity matrix computed on the combined data. In such cases where the graph is a discretization of an underlying Riemannian closed manifold, we prove that Diffusion EMD is topologically equivalent to the standard EMD with a geodesic ground distance. Diffusion EMD can be computed in $\\tilde{O}(n)$ time and is more accurate than similarly fast algorithms such as tree-based EMDs. We also show Diffusion EMD is fully differentiable, making it amenable to future uses in gradient-descent frameworks such as deep neural networks. Finally, we demonstrate an application of Diffusion EMD to single cell data collected from 210 COVID-19 patient samples at Yale New Haven Hospital. Here, Diffusion EMD can derive distances between patients on the manifold of cells at least two orders of magnitude faster than equally accurate methods. This distance matrix between patients can be embedded into a higher level patient manifold which uncovers structure and heterogeneity in patients. More generally, Diffusion EMD is applicable to all datasets that are massively collected in parallel in many medical and biological systems.","url_abs":"https://arxiv.org/abs/2102.12833v2","url_pdf":"https://arxiv.org/pdf/2102.12833v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusion-earth-mover-s-distance-and","repo_url":"https://github.com/KrishnaswamyLab/DiffusionEMD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.12833","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.12833"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KrishnaswamyLab/DiffusionEMD","reach":null}],"summary":{"ran":2,"ran_fixture":2,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"295bf82b8b736391","entry":"DiffusionEMD","repo":"KrishnaswamyLab/DiffusionEMD","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/KrishnaswamyLab/DiffusionEMD/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"295bf82b8b736391"}},{"code_sha256_prefix":"3324490bf67a556f","entry":"apply_anisotropy","repo":"KrishnaswamyLab/DiffusionEMD","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/KrishnaswamyLab/DiffusionEMD/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3324490bf67a556f"}},{"code_sha256_prefix":"962d224ce0612a7e","entry":"apply_vectors","repo":"KrishnaswamyLab/DiffusionEMD","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/KrishnaswamyLab/DiffusionEMD/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"962d224ce0612a7e"}},{"code_sha256_prefix":"8a5be38c36fa11e2","entry":"interpolative_decomposition","repo":"krishnaswamylab/diffusionemd","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/krishnaswamylab/diffusionemd/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8a5be38c36fa11e2"}},{"code_sha256_prefix":"c59720d5b8d2101e","entry":"approximate_rank","repo":"krishnaswamylab/diffusionemd","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/krishnaswamylab/diffusionemd/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c59720d5b8d2101e"}},{"code_sha256_prefix":"80a6eeb195ba9276","entry":"approximate_rank_of_scales","repo":"KrishnaswamyLab/DiffusionEMD","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/KrishnaswamyLab/DiffusionEMD/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"80a6eeb195ba9276"}},{"code_sha256_prefix":"7d2e3983fb523c61","entry":"estimate_dos","repo":"krishnaswamylab/diffusionemd","repo_kind":"official","path":"DiffusionEMD/diffusion_emd.py","file_url":"https://github.com/krishnaswamylab/diffusionemd/blob/HEAD/DiffusionEMD/diffusion_emd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7d2e3983fb523c61"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}