{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asformer-transformer-for-action-segmentation","title":"ASFormer: Transformer for Action Segmentation","arxiv_id":"2110.08568","date":"2021-10-16","proceeding":null,"authors":["Fangqiu Yi","Hongyu Wen","Tingting Jiang"],"abstract":"Algorithms for the action segmentation task typically use temporal models to predict what action is occurring at each frame for a minute-long daily activity. Recent studies have shown the potential of Transformer in modeling the relations among elements in sequential data. However, there are several major concerns when directly applying the Transformer to the action segmentation task, such as the lack of inductive biases with small training sets, the deficit in processing long input sequence, and the limitation of the decoder architecture to utilize temporal relations among multiple action segments to refine the initial predictions. To address these concerns, we design an efficient Transformer-based model for action segmentation task, named ASFormer, with three distinctive characteristics: (i) We explicitly bring in the local connectivity inductive priors because of the high locality of features. It constrains the hypothesis space within a reliable scope, and is beneficial for the action segmentation task to learn a proper target function with small training sets. (ii) We apply a pre-defined hierarchical representation pattern that efficiently handles long input sequences. (iii) We carefully design the decoder to refine the initial predictions from the encoder. Extensive experiments on three public datasets demonstrate that effectiveness of our methods. Code is available at \\url{https://github.com/ChinaYi/ASFormer}.","url_abs":"https://arxiv.org/abs/2110.08568v1","url_pdf":"https://arxiv.org/pdf/2110.08568v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asformer-transformer-for-action-segmentation","repo_url":"https://github.com/chinayi/asformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"action-segmentation","task_name":"Action Segmentation"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-segmentation-on-50-salads-1","task":"Action Segmentation","dataset":"50 Salads","model":"ASFormer+ASRF","rank_in_archive_order":12,"of":28,"metrics":{"Acc":"85.9","Edit":"81.9","F1@10%":"85.1","F1@25%":"85.4","F1@50%":"79.3"},"uses_additional_data":false},{"leaderboard":"/sota/action-segmentation-on-50-salads-1","task":"Action Segmentation","dataset":"50 Salads","model":"ASFormer","rank_in_archive_order":15,"of":28,"metrics":{"Acc":"85.6","Edit":"79.6","F1@10%":"85.1","F1@25%":"83.4","F1@50%":"76.0"},"uses_additional_data":false},{"leaderboard":"/sota/action-segmentation-on-assembly101","task":"Action Segmentation","dataset":"Assembly101","model":"ASFormer","rank_in_archive_order":3,"of":7,"metrics":{"Edit":"30.5","F1@10%":"33.4","F1@25%":"29.2","F1@50%":"21.4","MoF":"38.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-segmentation-on-breakfast-1","task":"Action Segmentation","dataset":"Breakfast","model":"ASFormer","rank_in_archive_order":13,"of":37,"metrics":{"Acc":"73.5","Average F1":"68.0","Edit":"75.0","F1@10%":"76.0","F1@25%":"70.6","F1@50%":"57.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-segmentation-on-gtea-1","task":"Action Segmentation","dataset":"GTEA","model":"ASFormer","rank_in_archive_order":12,"of":28,"metrics":{"Acc":"79.7","Edit":"84.6","F1@10%":"90.1","F1@25%":"88.8","F1@50%":"79.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.08568","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.08568"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chinayi/asformer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8467c56ed6949a06","entry":"exponential_descrease","repo":"chinayi/asformer","repo_kind":"official","path":"model.py","file_url":"https://github.com/chinayi/asformer/blob/HEAD/model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8467c56ed6949a06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}