{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/context-aware-sequence-alignment-using-4d","title":"Context-Aware Sequence Alignment using 4D Skeletal Augmentation","arxiv_id":"2204.12223","date":"2022-04-26","proceeding":"CVPR 2022 1","authors":["Taein Kwon","Bugra Tekin","Siyu Tang","Marc Pollefeys"],"abstract":"Temporal alignment of fine-grained human actions in videos is important for numerous applications in computer vision, robotics, and mixed reality. State-of-the-art methods directly learn image-based embedding space by leveraging powerful deep convolutional neural networks. While being straightforward, their results are far from satisfactory, the aligned videos exhibit severe temporal discontinuity without additional post-processing steps. The recent advancements in human body and hand pose estimation in the wild promise new ways of addressing the task of human action alignment in videos. In this work, based on off-the-shelf human pose estimators, we propose a novel context-aware self-supervised learning architecture to align sequences of actions. We name it CASA. Specifically, CASA employs self-attention and cross-attention mechanisms to incorporate the spatial and temporal context of human actions, which can solve the temporal discontinuity problem. Moreover, we introduce a self-supervised learning scheme that is empowered by novel 4D augmentation techniques for 3D skeleton representations. We systematically evaluate the key components of our method. Our experiments on three public datasets demonstrate CASA significantly improves phase progress and Kendall's Tau scores over the previous state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2204.12223v1","url_pdf":"https://arxiv.org/pdf/2204.12223v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"context-aware-sequence-alignment-using-4d","repo_url":"https://github.com/taeinkwon/CASA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"hand-pose-estimation","task_name":"Hand Pose Estimation"},{"task_slug":"mixed-reality","task_name":"Mixed Reality"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.12223","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.12223"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/taeinkwon/CASA","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"493d5a3eed4a09fe","entry":"Matching","repo":"taeinkwon/CASA","repo_kind":"listed","path":"src/casa/utils/matching.py","file_url":"https://github.com/taeinkwon/CASA/blob/HEAD/src/casa/utils/matching.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"493d5a3eed4a09fe"}},{"code_sha256_prefix":"addea8317843d1f8","entry":"dual_bicross","repo":"taeinkwon/CASA","repo_kind":"listed","path":"src/casa/utils/matching.py","file_url":"https://github.com/taeinkwon/CASA/blob/HEAD/src/casa/utils/matching.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"addea8317843d1f8"}},{"code_sha256_prefix":"4847f69947ac5174","entry":"dual_softmax","repo":"taeinkwon/CASA","repo_kind":"listed","path":"src/casa/utils/matching.py","file_url":"https://github.com/taeinkwon/CASA/blob/HEAD/src/casa/utils/matching.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4847f69947ac5174"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}