Papers › Fast Weakly Supervised Action Segmentation Using Mutual Consistency

Fast Weakly Supervised Action Segmentation Using Mutual Consistency

5 Apr 2019arXiv:1904.03116archive 2025-07-28

Yaser Souri, Mohsen Fayyaz, Luca Minciullo, Gianpiero Francesca, Juergen Gall

Action segmentation is the task of predicting the actions for each frame of a video. As obtaining the full annotation of videos for action segmentation is expensive, weakly supervised approaches that can learn only from transcripts are appealing. In this paper, we propose a novel end-to-end approach for weakly supervised action segmentation based on a two-branch neural network. The two branches of our network predict two redundant but different representations for action segmentation and we propose a novel mutual consistency (MuCon) loss that enforces the consistency of the two redundant representations. Using the MuCon loss together with a loss for transcript prediction, our proposed approach achieves the accuracy of state-of-the-art approaches while being $14$ times faster to train and $20$ times faster during inference. The MuCon loss proves beneficial even in the fully supervised setting.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

yassersouri/MuCon officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action Segmentation Segmentation Weakly Supervised Action Segmentation (Transcript)

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Action Segmentation	Breakfast	MuCon	Acc	62.8	#24 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	MuCon	Average F1	62.6	#24 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	MuCon	Edit	76.3	#24 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	MuCon	F1@10%	73.2	#24 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	MuCon	F1@25%	66.1	#24 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	MuCon	F1@50%	48.4	#24 of 37	Archive leaderboard	report
Weakly Supervised Action Segmentation (Transcript)	Breakfast	MuCon	Acc	48.5	#5 of 7	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1D CNN BiLSTM Dilated Convolution LSTM Sigmoid Activation Tanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections