</> Code with Papers by Syntology

Papers › Is Weakly-supervised Action Segmentation Ready For Human-Robot Interaction? No, Let's...

Is Weakly-supervised Action Segmentation Ready For Human-Robot Interaction? No, Let's Improve It With Action-union Learning

22 Oct 2023IROS 2023 2023 10archive 2025-07-28

Fan YangShigeyuki OdashimaShoichi MasuiShan Jiang

Action segmentation plays an important role in enabling robots to automatically understand human activities. To train the action recognition model, while obtaining action labels for all frames is costly, annotating timestamp labels for weak supervision is cost-effective. However, existing methods may not fully utilize timestamp labels, which leads to insufficient performance. To alleviate this issue, we proposed a novel learning pattern in our training stage, which maximizes the probability of action union of surrounding timestamps for unlabeled frames. In our inference stage, we provided a new refinement solution to generate better hard-assigned action classes from soft-assigned predictions. Importantly, our methods are model-agnostic and can be applied to existing frameworks. On three commonly used action-segmentation data, our method outperforms previous timestamp-supervision methods and achieves new state-of-the-art performance. Moreover , our method uses less than 1% of fully-supervised labels to obtain comparable or even better results.

PaperPDFCode

Code

fandulu/DD-Net tf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action RecognitionAction SegmentationSegmentationWeakly Supervised Action LocalizationWeakly Supervised Action Segmentation (Transcript)

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Action Segmentation 50 Salads AUL Acc 77.9 #24 of 28 Archive leaderboard report
Action Segmentation 50 Salads AUL Edit 77.0 #24 of 28 Archive leaderboard report
Action Segmentation 50 Salads AUL F1@10% 84.4 #24 of 28 Archive leaderboard report
Action Segmentation 50 Salads AUL F1@25% 81.3 #24 of 28 Archive leaderboard report
Action Segmentation 50 Salads AUL F1@50% 67.1 #24 of 28 Archive leaderboard report
Action Segmentation GTEA AUL Acc 69.2 #25 of 28 Archive leaderboard report
Action Segmentation GTEA AUL Edit 84.0 #25 of 28 Archive leaderboard report
Action Segmentation GTEA AUL F1@10% 88.2 #25 of 28 Archive leaderboard report
Action Segmentation GTEA AUL F1@25% 85.5 #25 of 28 Archive leaderboard report
Action Segmentation GTEA AUL F1@50% 67.3 #25 of 28 Archive leaderboard report
Weakly Supervised Action Localization GTEA AU-Action mAP@0.1:0.7 76.9 #1 of 6 Archive leaderboard report
Weakly Supervised Action Localization GTEA AU-Action mAP@0.5 66.3 #1 of 6 Archive leaderboard report
Weakly Supervised Action Segmentation (Transcript) Breakfast AUL Acc 67.3 #1 of 7 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections