Papers › ASQuery: A Query-based Model for Action Segmentation

ASQuery: A Query-based Model for Action Segmentation

30 Sep 20242024 IEEE International Conference on Multimedia and Expo (ICME) 2024 9archive 2025-07-28

Gan Ziliang, Jin Lei, Nie Lei, Zheng Wang, Li Zhou, Li Liang, Wang Zhecan, Li Jianshu, Xing Junliang, Zhao Jian

For the task of temporal action segmentation, existing works commonly treat it as a frame-wise classification problem. In this paper, we propose a straight but effective model namely ASQuery by learning central representation of each action category, which transforms the classification problem to the similarity calculation between category-specific queries and frame features. These central representations are dynamically generated through our Transformer decoder module, endowing them more flexible and comprehensive perception of the whole video. Moreover, we first introduce the boundary query for refining segmentation results, aiding to alleviating the troublesome over-segmentation problem. ASQuery demonstrates superior performance compared to state-of-the-art models, achieving improvements of 0.9% and 4.1% in the mean metrics on two public action segmentation datasets, i.e., Breakfast and Assembly101, respectively. The source codes are available at https://github.com/zlngan/ASQuery.

Paper PDF Code

Code

zlngan/ASQuery mentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action Segmentation Decoder Segmentation Temporal Action Segmentation model

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Action Segmentation	Assembly101	ASQuery	Edit	35.3	#1 of 7	Archive leaderboard	report
Action Segmentation	Assembly101	ASQuery	F1@10%	37.8	#1 of 7	Archive leaderboard	report
Action Segmentation	Assembly101	ASQuery	F1@25%	35.6	#1 of 7	Archive leaderboard	report
Action Segmentation	Assembly101	ASQuery	F1@50%	29.4	#1 of 7	Archive leaderboard	report
Action Segmentation	Assembly101	ASQuery	MoF	40.4	#1 of 7	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	Acc	77.9	#3 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	Average F1	74.6	#3 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	Edit	78.4	#3 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	F1@10%	80.7	#3 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	F1@25%	76.5	#3 of 37	Archive leaderboard	report
Action Segmentation	Breakfast	ASQuery	F1@50%	66.5	#3 of 37	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections