{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dyfadet-dynamic-feature-aggregation-for","title":"DyFADet: Dynamic Feature Aggregation for Temporal Action Detection","arxiv_id":"2407.03197","date":"2024-07-03","proceeding":null,"authors":["Le Yang","Ziwei Zheng","Yizeng Han","Hao Cheng","Shiji Song","Gao Huang","Fan Li"],"abstract":"Recent proposed neural network-based Temporal Action Detection (TAD) models are inherently limited to extracting the discriminative representations and modeling action instances with various lengths from complex scenes by shared-weights detection heads. Inspired by the successes in dynamic neural networks, in this paper, we build a novel dynamic feature aggregation (DFA) module that can simultaneously adapt kernel weights and receptive fields at different timestamps. Based on DFA, the proposed dynamic encoder layer aggregates the temporal features within the action time ranges and guarantees the discriminability of the extracted representations. Moreover, using DFA helps to develop a Dynamic TAD head (DyHead), which adaptively aggregates the multi-scale features with adjusted parameters and learned receptive fields better to detect the action instances with diverse ranges from videos. With the proposed encoder layer and DyHead, a new dynamic TAD model, DyFADet, achieves promising performance on a series of challenging TAD benchmarks, including HACS-Segment, THUMOS14, ActivityNet-1.3, Epic-Kitchen 100, Ego4D-Moment QueriesV1.0, and FineAction. Code is released to https://github.com/yangle15/DyFADet-pytorch.","url_abs":"https://arxiv.org/abs/2407.03197v1","url_pdf":"https://arxiv.org/pdf/2407.03197v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dyfadet-dynamic-feature-aggregation-for","repo_url":"https://github.com/yangle15/DyFADet-pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"dynamic-neural-networks","task_name":"Dynamic neural networks"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[{"method_slug":"dfa-1","method_name":"DFA"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/temporal-action-localization-on-fineaction","task":"Temporal Action Localization","dataset":"FineAction","model":"DyFADet (VideoMAE v2-g)","rank_in_archive_order":4,"of":9,"metrics":{"mAP":"23.8","mAP IOU@0.5":"37.1","mAP IOU@0.75":"23.7","mAP IOU@0.95":"5.9"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-action-localization-on-hacs","task":"Temporal Action Localization","dataset":"HACS","model":"DyFADet(VideoMAEv2)","rank_in_archive_order":3,"of":12,"metrics":{"Average-mAP":"44.3","mAP@0.5":"64.0","mAP@0.75":"44.8","mAP@0.95":"14.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.03197","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.03197"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangle15/DyFADet-pytorch","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran_honours":1,"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":12,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"a34c005ba2203f35","entry":"drop_path","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/blocks.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/blocks.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a34c005ba2203f35"}},{"code_sha256_prefix":"3531df7b0c9b0791","entry":"get_sinusoid_encoding","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/blocks.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/blocks.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3531df7b0c9b0791"}},{"code_sha256_prefix":"1033de592b951e93","entry":"load_config","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/core/config.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/core/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1033de592b951e93"}},{"code_sha256_prefix":"54293ea22bd4f952","entry":"sigmoid_focal_loss","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/losses.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54293ea22bd4f952"}},{"code_sha256_prefix":"02566da69866c48c","entry":"trunc_normal_","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/weight_init.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/weight_init.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02566da69866c48c"}},{"code_sha256_prefix":"fbd46ea8e0501851","entry":"truncate_feats","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/datasets/data_utils.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/datasets/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbd46ea8e0501851"}},{"code_sha256_prefix":"12d8ba65aabcb2b4","entry":"ctr_diou_loss_1d","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/losses.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12d8ba65aabcb2b4"}},{"code_sha256_prefix":"66693d03280664b1","entry":"ctr_giou_loss_1d","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/losses.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66693d03280664b1"}},{"code_sha256_prefix":"cee845f405ecf37e","entry":"register_backbone","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/models.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cee845f405ecf37e"}},{"code_sha256_prefix":"3db1c5ab5f3f0198","entry":"register_generator","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/models.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3db1c5ab5f3f0198"}},{"code_sha256_prefix":"e5ef13b176081594","entry":"register_neck","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/modeling/models.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/modeling/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5ef13b176081594"}},{"code_sha256_prefix":"0c250bf03b8c9311","entry":"trivial_batch_collator","repo":"yangle15/DyFADet-pytorch","repo_kind":"official","path":"libs/datasets/data_utils.py","file_url":"https://github.com/yangle15/DyFADet-pytorch/blob/HEAD/libs/datasets/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c250bf03b8c9311"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}