{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/activity-grammars-for-temporal-action-1","title":"Activity Grammars for Temporal Action Segmentation","arxiv_id":"2312.04266","date":"2023-12-07","proceeding":"NeurIPS 2023 11","authors":["Dayoung Gong","Joonseok Lee","Deunsol Jung","Suha Kwak","Minsu Cho"],"abstract":"Sequence prediction on temporal data requires the ability to understand compositional structures of multi-level semantics beyond individual and contextual properties. The task of temporal action segmentation, which aims at translating an untrimmed activity video into a sequence of action segments, remains challenging for this reason. This paper addresses the problem by introducing an effective activity grammar to guide neural predictions for temporal action segmentation. We propose a novel grammar induction algorithm that extracts a powerful context-free grammar from action sequence data. We also develop an efficient generalized parser that transforms frame-level probability distributions into a reliable sequence of actions according to the induced grammar with recursive rules. Our approach can be combined with any neural network for temporal action segmentation to enhance the sequence prediction and discover its compositional structure. Experimental results demonstrate that our method significantly improves temporal action segmentation in terms of both performance and interpretability on two standard benchmarks, Breakfast and 50 Salads.","url_abs":"https://arxiv.org/abs/2312.04266v1","url_pdf":"https://arxiv.org/pdf/2312.04266v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"activity-grammars-for-temporal-action-1","repo_url":"https://github.com/gongda0e/kari","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-segmentation","task_name":"Action Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"temporal-action-segmentation","task_name":"Temporal Action Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.04266","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.04266"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gongda0e/kari","reach":null}],"summary":{"ran_draft_wrong":3,"ran":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"43c9f4ee54cbfa4e","entry":"state","repo":"gongda0e/kari","repo_kind":"official","path":"kari.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/kari.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43c9f4ee54cbfa4e"}},{"code_sha256_prefix":"a5a7ca180bd4ca14","entry":"State","repo":"gongda0e/kari","repo_kind":"official","path":"bep.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/bep.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5a7ca180bd4ca14"}},{"code_sha256_prefix":"99b12cd7e43a02eb","entry":"concat_seqs_to_seqs","repo":"gongda0e/kari","repo_kind":"official","path":"kari.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/kari.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99b12cd7e43a02eb"}},{"code_sha256_prefix":"17b55e70208dd3ca","entry":"format_num","repo":"gongda0e/kari","repo_kind":"official","path":"bep.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/bep.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17b55e70208dd3ca"}},{"code_sha256_prefix":"14303f0e90cccc3b","entry":"remove_consecutive","repo":"gongda0e/kari","repo_kind":"official","path":"kari.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/kari.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14303f0e90cccc3b"}},{"code_sha256_prefix":"5c537cba3835d120","entry":"BreadthFirstEarley","repo":"gongda0e/kari","repo_kind":"official","path":"bep.py","file_url":"https://github.com/gongda0e/kari/blob/HEAD/bep.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c537cba3835d120"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}