{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tuber-tube-transformer-for-action-detection","title":"TubeR: Tubelet Transformer for Video Action Detection","arxiv_id":"2104.00969","date":"2021-04-02","proceeding":"CVPR 2022 1","authors":["Jiaojiao Zhao","Yanyi Zhang","Xinyu Li","Hao Chen","Shuai Bing","Mingze Xu","Chunhui Liu","Kaustav Kundu","Yuanjun Xiong","Davide Modolo","Ivan Marsic","Cees G. M. Snoek","Joseph Tighe"],"abstract":"We propose TubeR: a simple solution for spatio-temporal video action detection. Different from existing methods that depend on either an off-line actor detector or hand-designed actor-positional hypotheses like proposals or anchors, we propose to directly detect an action tubelet in a video by simultaneously performing action localization and recognition from a single representation. TubeR learns a set of tubelet-queries and utilizes a tubelet-attention module to model the dynamic spatio-temporal nature of a video clip, which effectively reinforces the model capacity compared to using actor-positional hypotheses in the spatio-temporal space. For videos containing transitional states or scene changes, we propose a context aware classification head to utilize short-term and long-term context to strengthen action classification, and an action switch regression head for detecting the precise temporal action extent. TubeR directly produces action tubelets with variable lengths and even maintains good results for long video clips. TubeR outperforms the previous state-of-the-art on commonly used action detection datasets AVA, UCF101-24 and JHMDB51-21.","url_abs":"https://arxiv.org/abs/2104.00969v5","url_pdf":"https://arxiv.org/pdf/2104.00969v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tuber-tube-transformer-for-action-detection","repo_url":"https://github.com/amazon-science/tubelet-transformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"action-localization","task_name":"Action Localization"},{"task_slug":"video-action-detection","task_name":"Video Action Detection"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[{"method_slug":"aware","method_name":"AWARE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2104.00969","atlas_url":"https://app.syntology.ai/?focus=2104.00969","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.00969"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amazon-science/tubelet-transformer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran_honours":3,"ran":2,"unverified":2},"by_repo_kind":{"listed":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aa6da43d05cfaf65","entry":"all_gather","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/misc.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/misc.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa6da43d05cfaf65"}},{"code_sha256_prefix":"e0a06ded5d4f6c3c","entry":"box_cxcywh_to_xyxy","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/box_ops.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0a06ded5d4f6c3c"}},{"code_sha256_prefix":"ce749f837424dd8c","entry":"box_iou","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/box_ops.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce749f837424dd8c"}},{"code_sha256_prefix":"031b02fb990284ba","entry":"box_xyxy_to_cxcywh","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/box_ops.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"031b02fb990284ba"}},{"code_sha256_prefix":"dc78bab96f28fdc8","entry":"collate_fn","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/misc.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dc78bab96f28fdc8"}},{"code_sha256_prefix":"99c111ef29c683f2","entry":"layer_norm","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/transformer_layers.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/transformer_layers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99c111ef29c683f2"}},{"code_sha256_prefix":"05992505df9ef0cd","entry":"reduce_dict","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/misc.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/misc.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"05992505df9ef0cd"}},{"code_sha256_prefix":"7b94aa61b4d5ecac","entry":"build_transformer","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/transformer.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b94aa61b4d5ecac"}},{"code_sha256_prefix":"fe699ae8220daf07","entry":"plot_precision_recall","repo":"amazon-science/tubelet-transformer","repo_kind":"listed","path":"models/transformer/util/plot_utils.py","file_url":"https://github.com/amazon-science/tubelet-transformer/blob/HEAD/models/transformer/util/plot_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe699ae8220daf07"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}