Papers › Progress-Aware Online Action Segmentation for Egocentric Procedural Task Videos
Progress-Aware Online Action Segmentation for Egocentric Procedural Task Videos
YuHan Shen, Ehsan Elhamifar
We address the problem of online action segmentation for egocentric procedural task videos. While previous studies have mostly focused on offline action segmentation where entire videos are available for both training and inference the transition to online action segmentation is crucial for practical applications like AR/VR task assistants. Notably applying an offline-trained model directly to online inference results in a significant performance drop due to the inconsistency between training and inference. We propose an online action segmentation framework by first modifying existing architectures to make them causal. Second we develop a novel action progress prediction module to dynamically estimate the progress of ongoing actions and using them to refine the predictions of causal action segmentation. Third we propose to learn task graphs from training videos and leverage them to obtain smooth and procedure-consistent segmentations. With the combination of progress and task graph with casual action segmentation our framework effectively addresses prediction uncertainty and oversegmentation in online action segmentation and achieves significant improvement on three egocentric datasets.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Segmentation | Assembly101 | ProTAS(Offline) | Edit | 29.2 | #7 of 7 | Archive leaderboard | report |
| Action Segmentation | Assembly101 | ProTAS(Offline) | F1@10% | 28.7 | #7 of 7 | Archive leaderboard | report |
| Action Segmentation | Assembly101 | ProTAS(Offline) | F1@25% | 24.4 | #7 of 7 | Archive leaderboard | report |
| Action Segmentation | Assembly101 | ProTAS(Offline) | F1@50% | 17.5 | #7 of 7 | Archive leaderboard | report |
| Action Segmentation | Assembly101 | ProTAS(Offline) | MoF | 34.8 | #7 of 7 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections