Papers › Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs

Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs

9 Jan 2016CVPR 2016 6arXiv:1601.02129archive 2025-07-28

Zheng Shou, Dongang Wang, Shih-Fu Chang

We address temporal action localization in untrimmed long videos. This is important because videos in real applications are usually unconstrained and contain multiple action instances plus video content of background scenes or other activities. To address this challenging issue, we exploit the effectiveness of deep networks in temporal action localization via three segment-based 3D ConvNets: (1) a proposal network identifies candidate segments in a long video that may contain actions; (2) a classification network learns one-vs-all action classification model to serve as initialization for the localization network; and (3) a localization network fine-tunes on the learned classification network to localize each action instance. We propose a novel loss function for the localization network to explicitly consider temporal overlap and therefore achieve high temporal localization accuracy. Only the proposal network and the localization network are used during prediction. On two large-scale benchmarks, our approach achieves significantly superior performances compared with other state-of-the-art systems: mAP increases from 1.7% to 7.4% on MEXaction2 and increases from 15.0% to 19.0% on THUMOS 2014, when the overlap threshold for evaluation is set to 0.5.

Paper PDF Conference PDF Code

Code

zhengshou/scnn officialmentioned in paper report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action Classification Action Localization Classification General Classification Temporal Action Localization Temporal Localization

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Action Recognition	THUMOS’14	Shou et. al.	mAP@0.1	47.7	#9 of 10	Archive leaderboard	report
Action Recognition	THUMOS’14	Shou et. al.	mAP@0.2	43.5	#9 of 10	Archive leaderboard	report
Action Recognition	THUMOS’14	Shou et. al.	mAP@0.3	36.3	#9 of 10	Archive leaderboard	report
Action Recognition	THUMOS’14	Shou et. al.	mAP@0.4	28.7	#9 of 10	Archive leaderboard	report
Action Recognition	THUMOS’14	Shou et. al.	mAP@0.5	19.0	#9 of 10	Archive leaderboard	report
Temporal Action Localization	MEXaction2	S-CNN	mAP	7.4	#1 of 1	Archive leaderboard	report
Temporal Action Localization	THUMOS’14	S-CNN	mAP IOU@0.1	47.7	#38 of 42	Archive leaderboard	report
Temporal Action Localization	THUMOS’14	S-CNN	mAP IOU@0.2	43.5	#38 of 42	Archive leaderboard	report
Temporal Action Localization	THUMOS’14	S-CNN	mAP IOU@0.3	36.3	#38 of 42	Archive leaderboard	report
Temporal Action Localization	THUMOS’14	S-CNN	mAP IOU@0.4	28.7	#38 of 42	Archive leaderboard	report
Temporal Action Localization	THUMOS’14	S-CNN	mAP IOU@0.5	19	#38 of 42	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections