{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/representation-learning-via-global-temporal","title":"Representation Learning via Global Temporal Alignment and Cycle-Consistency","arxiv_id":"2105.05217","date":"2021-05-11","proceeding":"CVPR 2021 1","authors":["Isma Hadji","Konstantinos G. Derpanis","Allan D. Jepson"],"abstract":"We introduce a weakly supervised method for representation learning based on aligning temporal sequences (e.g., videos) of the same process (e.g., human action). The main idea is to use the global temporal ordering of latent correspondences across sequence pairs as a supervisory signal. In particular, we propose a loss based on scoring the optimal sequence alignment to train an embedding network. Our loss is based on a novel probabilistic path finding view of dynamic time warping (DTW) that contains the following three key features: (i) the local path routing decisions are contrastive and differentiable, (ii) pairwise distances are cast as probabilities that are contrastive as well, and (iii) our formulation naturally admits a global cycle consistency loss that verifies correspondences. For evaluation, we consider the tasks of fine-grained action classification, few shot learning, and video synchronization. We report significant performance increases over previous methods. In addition, we report two applications of our temporal alignment framework, namely 3D pose reconstruction and fine-grained audio/visual retrieval.","url_abs":"https://arxiv.org/abs/2105.05217v1","url_pdf":"https://arxiv.org/pdf/2105.05217v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"representation-learning-via-global-temporal","repo_url":"https://github.com/hadjisma/VideoAlignment","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"dynamic-time-warping","task_name":"Dynamic Time Warping"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"temporal-sequences","task_name":"Temporal Sequences"},{"task_slug":"video-synchronization","task_name":"Video Synchronization"}],"methods":[{"method_slug":"cycle-consistency-loss","method_name":"Cycle Consistency Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2105.05217","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.05217"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hadjisma/VideoAlignment","reach":null}],"summary":{"ran_fixture":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"7655bf9f79516b33","entry":"DTW_loss","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7655bf9f79516b33"}},{"code_sha256_prefix":"52a25c6aa4b5d90a","entry":"assign2Tensor","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"52a25c6aa4b5d90a"}},{"code_sha256_prefix":"e3d8c79482d57679","entry":"classification_loss","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e3d8c79482d57679"}},{"code_sha256_prefix":"4a7b509ff2864f8b","entry":"minGamma","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4a7b509ff2864f8b"}},{"code_sha256_prefix":"f7b76021e9bd4372","entry":"compute_dtw_alignment_consistency_loss","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f7b76021e9bd4372"}},{"code_sha256_prefix":"170a08663de92be4","entry":"smoothDTW","repo":"hadjisma/VideoAlignment","repo_kind":"official","path":"d2tw/smoothDTW.py","file_url":"https://github.com/hadjisma/VideoAlignment/blob/HEAD/d2tw/smoothDTW.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"170a08663de92be4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}