Papers › Enhancing Temporal Action Localization: Advanced S6 Modeling with Recurrent Mechanism

Enhancing Temporal Action Localization: Advanced S6 Modeling with Recurrent Mechanism

18 Jul 2024arXiv:2407.13078archive 2025-07-28

Sangyoun Lee, Juho Jung, Changdae Oh, Sunghee Yun

Temporal Action Localization (TAL) is a critical task in video analysis, identifying precise start and end times of actions. Existing methods like CNNs, RNNs, GCNs, and Transformers have limitations in capturing long-range dependencies and temporal causality. To address these challenges, we propose a novel TAL architecture leveraging the Selective State Space Model (S6). Our approach integrates the Feature Aggregated Bi-S6 block, Dual Bi-S6 structure, and a recurrent mechanism to enhance temporal and channel-wise dependency modeling without increasing parameter complexity. Extensive experiments on benchmark datasets demonstrate state-of-the-art results with mAP scores of 74.2% on THUMOS-14, 42.9% on ActivityNet, 29.6% on FineAction, and 45.8% on HACS. Ablation studies validate our method's effectiveness, showing that the Dual structure in the Stem module and the recurrent mechanism outperform traditional approaches. Our findings demonstrate the potential of S6-based models in TAL tasks, paving the way for future research.

PaperPDFCode

Code

lsy0882/RDFA-S6 officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action LocalizationTemporal Action Localization

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Temporal Action Localization ActivityNet-1.3 RDFA-S6 (InternVideo2-6B) mAP 42.9 #1 of 33 Archive leaderboard report
Temporal Action Localization ActivityNet-1.3 RDFA-S6 (InternVideo2-6B) mAP IOU@0.5 64.1 #1 of 33 Archive leaderboard report
Temporal Action Localization ActivityNet-1.3 RDFA-S6 (InternVideo2-6B) mAP IOU@0.75 44.0 #1 of 33 Archive leaderboard report
Temporal Action Localization ActivityNet-1.3 RDFA-S6 (InternVideo2-6B) mAP IOU@0.95 10.6 #1 of 33 Archive leaderboard report
Temporal Action Localization FineAction RDFA-S6 (InternVideo2-6B) mAP 29.6 #1 of 9 Archive leaderboard report
Temporal Action Localization FineAction RDFA-S6 (InternVideo2-6B) mAP IOU@0.5 46.4 #1 of 9 Archive leaderboard report
Temporal Action Localization FineAction RDFA-S6 (InternVideo2-6B) mAP IOU@0.75 29.5 #1 of 9 Archive leaderboard report
Temporal Action Localization FineAction RDFA-S6 (InternVideo2-6B) mAP IOU@0.95 7.6 #1 of 9 Archive leaderboard report
Temporal Action Localization HACS RDFA-S6 (InternVideo2-6B) Average-mAP 45.8 #1 of 12 Archive leaderboard report
Temporal Action Localization HACS RDFA-S6 (InternVideo2-6B) mAP@0.5 66.4 #1 of 12 Archive leaderboard report
Temporal Action Localization HACS RDFA-S6 (InternVideo2-6B) mAP@0.75 47.2 #1 of 12 Archive leaderboard report
Temporal Action Localization HACS RDFA-S6 (InternVideo2-6B) mAP@0.95 14.3 #1 of 12 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) Avg mAP (0.3:0.7) 74.2 #2 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) mAP IOU@0.3 88.7 #2 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) mAP IOU@0.4 84.6 #2 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) mAP IOU@0.5 78.2 #2 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) mAP IOU@0.6 66.6 #2 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 RDFA-S6 (InternVideo2-6B) mAP IOU@0.7 51.9 #2 of 42 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections