{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-the-robustness-of-temporal","title":"Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions","arxiv_id":"2403.20254","date":"2024-03-29","proceeding":"CVPR 2024 1","authors":["Runhao Zeng","Xiaoyong Chen","Jiaming Liang","Huisi Wu","Guangzhong Cao","Yong Guo"],"abstract":"Temporal action detection (TAD) aims to locate action positions and recognize action categories in long-term untrimmed videos. Although many methods have achieved promising results, their robustness has not been thoroughly studied. In practice, we observe that temporal information in videos can be occasionally corrupted, such as missing or blurred frames. Interestingly, existing methods often incur a significant performance drop even if only one frame is affected. To formally evaluate the robustness, we establish two temporal corruption robustness benchmarks, namely THUMOS14-C and ActivityNet-v1.3-C. In this paper, we extensively analyze the robustness of seven leading TAD methods and obtain some interesting findings: 1) Existing methods are particularly vulnerable to temporal corruptions, and end-to-end methods are often more susceptible than those with a pre-trained feature extractor; 2) Vulnerability mainly comes from localization error rather than classification error; 3) When corruptions occur in the middle of an action instance, TAD models tend to yield the largest performance drop. Besides building a benchmark, we further develop a simple but effective robust training method to defend against temporal corruptions, through the FrameDrop augmentation and Temporal-Robust Consistency loss. Remarkably, our approach not only improves robustness but also yields promising improvements on clean data. We believe that this study will serve as a benchmark for future research in robust video analysis. Source code and models are available at https://github.com/Alvin-Zeng/temporal-robustness-benchmark.","url_abs":"https://arxiv.org/abs/2403.20254v1","url_pdf":"https://arxiv.org/pdf/2403.20254v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-the-robustness-of-temporal","repo_url":"https://github.com/alvin-zeng/temporal-robustness-benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.20254","atlas_url":"https://app.syntology.ai/?focus=2403.20254","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.20254"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran":6,"unverified":9},"by_repo_kind":{"official":{"samples":16,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"da651e3979a18f84","entry":"get_sinusoid_encoding_table","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/models/modeling_finetune.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/models/modeling_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da651e3979a18f84"}},{"code_sha256_prefix":"64c14fdce33f430d","entry":"get_start_idx_range","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64c14fdce33f430d"}},{"code_sha256_prefix":"ecd670b7a5bb15ed","entry":"get_start_idx_range","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/extract_tad_feature_thumos14.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/extract_tad_feature_thumos14.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ecd670b7a5bb15ed"}},{"code_sha256_prefix":"a8ccf61e0ee23809","entry":"load_json","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/libs/pre_process_data_anet.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/libs/pre_process_data_anet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8ccf61e0ee23809"}},{"code_sha256_prefix":"024894a396cde124","entry":"multiple_samples_collate","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/utils.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"024894a396cde124"}},{"code_sha256_prefix":"8d1dc1aa52fe2112","entry":"resize","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d1dc1aa52fe2112"}},{"code_sha256_prefix":"eab677b96697e36b","entry":"to_normalized_float_tensor","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/copy_extract_tad_feature.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eab677b96697e36b"}},{"code_sha256_prefix":"283de3b7dcb770a3","entry":"build_dataset","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/dataset/build.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/dataset/build.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"283de3b7dcb770a3"}},{"code_sha256_prefix":"cb5e3e409ec627d3","entry":"build_pretraining_dataset","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/dataset/build.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/dataset/build.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb5e3e409ec627d3"}},{"code_sha256_prefix":"b700fb0a3496b255","entry":"cosine_scheduler","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/utils.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b700fb0a3496b255"}},{"code_sha256_prefix":"ba1356e8ceb654d2","entry":"get_grad_norm_","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/utils.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba1356e8ceb654d2"}},{"code_sha256_prefix":"29416043c7035e7c","entry":"get_loss_scale_for_deepspeed","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/engine_for_finetuning.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/engine_for_finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29416043c7035e7c"}},{"code_sha256_prefix":"a0ba0d4e39b8caf0","entry":"pre_process_annotations_data","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/libs/pre_process_data_anet.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/libs/pre_process_data_anet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0ba0d4e39b8caf0"}},{"code_sha256_prefix":"7e49117a60f3b108","entry":"spatial_sampling","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/dataset/datasets.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/dataset/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e49117a60f3b108"}},{"code_sha256_prefix":"14b3b13b2e7ee844","entry":"tensor_normalize","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/dataset/datasets.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/dataset/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"14b3b13b2e7ee844"}},{"code_sha256_prefix":"c7976ea27edc377a","entry":"train_class_batch","repo":"Alvin-Zeng/temporal-robustness-benchmark","repo_kind":"official","path":"extract_corrupted_feature_code/videomae_v2/engine_for_finetuning.py","file_url":"https://github.com/Alvin-Zeng/temporal-robustness-benchmark/blob/HEAD/extract_corrupted_feature_code/videomae_v2/engine_for_finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7976ea27edc377a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}