{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mosic-optimal-transport-motion-trajectory-for","title":"MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning","arxiv_id":"2506.08694","date":"2025-06-10","proceeding":null,"authors":["Mohammadreza Salehi","Shashanka Venkataramanan","Ioana Simion","Efstratios Gavves","Cees G. M. Snoek","Yuki M Asano"],"abstract":"Dense self-supervised learning has shown great promise for learning pixel- and patch-level representations, but extending it to videos remains challenging due to the complexity of motion dynamics. Existing approaches struggle as they rely on static augmentations that fail under object deformations, occlusions, and camera movement, leading to inconsistent feature learning over time. We propose a motion-guided self-supervised learning framework that clusters dense point tracks to learn spatiotemporally consistent representations. By leveraging an off-the-shelf point tracker, we extract long-range motion trajectories and optimize feature clustering through a momentum-encoder-based optimal transport mechanism. To ensure temporal coherence, we propagate cluster assignments along tracked points, enforcing feature consistency across views despite viewpoint changes. Integrating motion as an implicit supervisory signal, our method learns representations that generalize across frames, improving robustness in dynamic scenes and challenging occlusion scenarios. By initializing from strong image-pretrained models and leveraging video data for training, we improve state-of-the-art by 1% to 6% on six image and video datasets and four evaluation benchmarks. The implementation is publicly available at our GitHub repository: https://github.com/SMSD75/MoSiC/tree/main","url_abs":"https://arxiv.org/abs/2506.08694v1","url_pdf":"https://arxiv.org/pdf/2506.08694v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mosic-optimal-transport-motion-trajectory-for","repo_url":"https://github.com/smsd75/mosic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2506.08694","atlas_url":"https://app.syntology.ai/?focus=2506.08694","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.08694"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/smsd75/mosic","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/SMSD75/MoSiC","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":4,"ran_draft_wrong":3,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"4200c34fe255995e","entry":"FeatureExtractor","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4200c34fe255995e"}},{"code_sha256_prefix":"a50c7d76e997813d","entry":"FeatureForwarder","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a50c7d76e997813d"}},{"code_sha256_prefix":"c6297d113584e84a","entry":"FixedMaskPatchDropout","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c6297d113584e84a"}},{"code_sha256_prefix":"dbccedda58a1875b","entry":"LearnableUpsampler","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"dbccedda58a1875b"}},{"code_sha256_prefix":"a5ef08c2d4538b2b","entry":"denormalize_video","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a5ef08c2d4538b2b"}},{"code_sha256_prefix":"036a8b9f1d3b71e5","entry":"find_optimal_assignment","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"036a8b9f1d3b71e5"}},{"code_sha256_prefix":"432bf71f80b71af8","entry":"sinkhorn","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"432bf71f80b71af8"}},{"code_sha256_prefix":"10764c5f33366319","entry":"MoSiC","repo":"SMSD75/MoSiC","repo_kind":"official","path":"experiments/exp_mosic.py","file_url":"https://github.com/SMSD75/MoSiC/blob/HEAD/experiments/exp_mosic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"10764c5f33366319"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}