{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vpn-rethinking-video-pose-embeddings-for","title":"VPN++: Rethinking Video-Pose embeddings for understanding Activities of Daily Living","arxiv_id":"2105.08141","date":"2021-05-17","proceeding":null,"authors":["Srijan Das","Rui Dai","Di Yang","Francois Bremond"],"abstract":"Many attempts have been made towards combining RGB and 3D poses for the recognition of Activities of Daily Living (ADL). ADL may look very similar and often necessitate to model fine-grained details to distinguish them. Because the recent 3D ConvNets are too rigid to capture the subtle visual patterns across an action, this research direction is dominated by methods combining RGB and 3D Poses. But the cost of computing 3D poses from RGB stream is high in the absence of appropriate sensors. This limits the usage of aforementioned approaches in real-world applications requiring low latency. Then, how to best take advantage of 3D Poses for recognizing ADL? To this end, we propose an extension of a pose driven attention mechanism: Video-Pose Network (VPN), exploring two distinct directions. One is to transfer the Pose knowledge into RGB through a feature-level distillation and the other towards mimicking pose driven attention through an attention-level distillation. Finally, these two approaches are integrated into a single model, we call VPN++. We show that VPN++ is not only effective but also provides a high speed up and high resilience to noisy Poses. VPN++, with or without 3D Poses, outperforms the representative baselines on 4 public datasets. Code is available at https://github.com/srijandas07/vpnplusplus.","url_abs":"https://arxiv.org/abs/2105.08141v1","url_pdf":"https://arxiv.org/pdf/2105.08141v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vpn-rethinking-video-pose-embeddings-for","repo_url":"https://github.com/srijandas07/vpnplusplus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"skeleton-based-action-recognition","task_name":"Skeleton Based Action Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-recognition-in-videos-on-ntu-rgbd-120","task":"Action Recognition","dataset":"NTU RGB+D 120","model":"VPN++ (RGB + Pose)","rank_in_archive_order":13,"of":21,"metrics":{"Accuracy (Cross-Setup)":"90.7","Accuracy (Cross-Subject)":"92.5"},"uses_additional_data":true},{"leaderboard":"/sota/skeleton-based-action-recognition-on-n-ucla","task":"Skeleton Based Action Recognition","dataset":"N-UCLA","model":"VPN++ (RGB + Pose)","rank_in_archive_order":20,"of":25,"metrics":{"Accuracy":"93.5"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2105.08141","atlas_url":"https://app.syntology.ai/?focus=2105.08141","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.08141"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/srijandas07/vpnplusplus","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"ffed4f85d50832c9","entry":"import_class","repo":"srijandas07/vpnplusplus","repo_kind":"official","path":"model/aagcn.py","file_url":"https://github.com/srijandas07/vpnplusplus/blob/HEAD/model/aagcn.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffed4f85d50832c9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}