</> Code with Papers by Syntology

Papers › End-to-end Learning of Action Detection from Frame Glimpses in Videos

End-to-end Learning of Action Detection from Frame Glimpses in Videos

22 Nov 2015CVPR 2016 6arXiv:1511.06984archive 2025-07-28

Serena Yeung, Olga Russakovsky, Greg Mori, Li Fei-Fei

In this work we introduce a fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions. Our intuition is that the process of detecting actions is naturally one of observation and refinement: observing moments in video, and refining hypotheses about when an action is occurring. Based on this insight, we formulate our model as a recurrent neural network-based agent that interacts with a video over time. The agent observes video frames and decides both where to look next and when to emit a prediction. Since backpropagation is not adequate in this non-differentiable setting, we use REINFORCE to learn the agent's decision policy. Our model achieves state-of-the-art results on the THUMOS'14 and ActivityNet datasets while observing only a fraction (2% or less) of the video frames.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

syyeung/frameglimpses mentioned on GitHubtorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action DetectionTemporal Action Localization

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Action Recognition THUMOS’14 Yeung et. al. mAP@0.1 48.9 #10 of 10 Archive leaderboard report
Action Recognition THUMOS’14 Yeung et. al. mAP@0.2 44.0 #10 of 10 Archive leaderboard report
Action Recognition THUMOS’14 Yeung et. al. mAP@0.3 36.0 #10 of 10 Archive leaderboard report
Action Recognition THUMOS’14 Yeung et. al. mAP@0.4 26.4 #10 of 10 Archive leaderboard report
Action Recognition THUMOS’14 Yeung et. al. mAP@0.5 17.1 #10 of 10 Archive leaderboard report
Temporal Action Localization THUMOS’14 Yeung et al. mAP IOU@0.1 48.9 #37 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 Yeung et al. mAP IOU@0.2 44.0 #37 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 Yeung et al. mAP IOU@0.3 36.0 #37 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 Yeung et al. mAP IOU@0.4 26.4 #37 of 42 Archive leaderboard report
Temporal Action Localization THUMOS’14 Yeung et al. mAP IOU@0.5 17.1 #37 of 42 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

REINFORCE

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections