{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ms-tct-multi-scale-temporal-convtransformer","title":"MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection","arxiv_id":"2112.03902","date":"2021-12-07","proceeding":"CVPR 2022 1","authors":["Rui Dai","Srijan Das","Kumara Kahatapitiya","Michael S. Ryoo","Francois Bremond"],"abstract":"Action detection is an essential and challenging task, especially for densely labelled datasets of untrimmed videos. The temporal relation is complex in those datasets, including challenges like composite action, and co-occurring action. For detecting actions in those complex videos, efficiently capturing both short-term and long-term temporal information in the video is critical. To this end, we propose a novel ConvTransformer network for action detection. This network comprises three main components: (1) Temporal Encoder module extensively explores global and local temporal relations at multiple temporal resolutions. (2) Temporal Scale Mixer module effectively fuses the multi-scale features to have a unified feature representation. (3) Classification module is used to learn the instance center-relative position and predict the frame-level classification scores. The extensive experiments on multiple datasets, including Charades, TSU and MultiTHUMOS, confirm the effectiveness of our proposed method. Our network outperforms the state-of-the-art methods on all three datasets.","url_abs":"https://arxiv.org/abs/2112.03902v2","url_pdf":"https://arxiv.org/pdf/2112.03902v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ms-tct-multi-scale-temporal-convtransformer","repo_url":"https://github.com/dairui01/MS-TCT","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-detection-on-charades","task":"Action Detection","dataset":"Charades","model":"MS-TCT (RGB only)","rank_in_archive_order":7,"of":16,"metrics":{"mAP":"25.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-detection-on-multi-thumos","task":"Action Detection","dataset":"Multi-THUMOS","model":"MS-TCT (RGB only)","rank_in_archive_order":5,"of":8,"metrics":{"mAP":"43.1"},"uses_additional_data":false},{"leaderboard":"/sota/action-detection-on-tsu","task":"Action Detection","dataset":"TSU","model":"MS-TCT","rank_in_archive_order":2,"of":2,"metrics":{"Frame-mAP":"33.7"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-action-localization-on-multithumos-1","task":"Temporal Action Localization","dataset":"MultiTHUMOS","model":"MS-TCT","rank_in_archive_order":7,"of":8,"metrics":{"Average mAP":"16.2"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2112.03902","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.03902"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dairui01/MS-TCT","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"named_in_paper":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"4d4c6a1743edf270","entry":"make_gt","repo":"dairui01/MS-TCT","repo_kind":"named_in_paper","path":"Evaluation.py","file_url":"https://github.com/dairui01/MS-TCT/blob/HEAD/Evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d4c6a1743edf270"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}