Papers › Improving Action Segmentation via Graph-Based Temporal Reasoning
Improving Action Segmentation via Graph-Based Temporal Reasoning
Yifei Huang, Yusuke Sugano, Yoichi Sato
Temporal relations among multiple action segments play an important role in action segmentation especially when observations are limited (e.g., actions are occluded by other objects or happen outside a field of view). In this paper, we propose a network module called Graph-based Temporal Reasoning Module (GTRM) that can be built on top of existing action segmentation models to learn the relation of multiple action segments in various time spans. We model the relations by using two Graph Convolution Networks (GCNs) where each node represents an action segment. The two graphs have different edge properties to account for boundary regression and classification tasks, respectively. By applying graph convolution, we can update each node's representation based on its relation with neighboring nodes. The updated representation is then used for improved action segmentation. We evaluate our model on the challenging egocentric datasets namely EGTEA and EPIC-Kitchens, where actions may be partially observed due to the viewpoint restriction. The results show that our proposed GTRM outperforms state-of-the-art action segmentation models by a large margin. We also demonstrate the effectiveness of our model on two third-person video datasets, the 50Salads dataset and the Breakfast dataset.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Segmentation | Breakfast | GTRM | Acc | 65.0 | #29 of 37 | Archive leaderboard | report |
| Action Segmentation | Breakfast | GTRM | Average F1 | 51.6 | #29 of 37 | Archive leaderboard | report |
| Action Segmentation | Breakfast | GTRM | Edit | 58.7 | #29 of 37 | Archive leaderboard | report |
| Action Segmentation | Breakfast | GTRM | F1@10% | 57.5 | #29 of 37 | Archive leaderboard | report |
| Action Segmentation | Breakfast | GTRM | F1@25% | 54.0 | #29 of 37 | Archive leaderboard | report |
| Action Segmentation | Breakfast | GTRM | F1@50% | 43.3 | #29 of 37 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections