{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tallformer-temporal-action-localization-with","title":"TALLFormer: Temporal Action Localization with a Long-memory Transformer","arxiv_id":"2204.01680","date":"2022-04-04","proceeding":null,"authors":["Feng Cheng","Gedas Bertasius"],"abstract":"Most modern approaches in temporal action localization divide this problem into two parts: (i) short-term feature extraction and (ii) long-range temporal boundary localization. Due to the high GPU memory cost caused by processing long untrimmed videos, many methods sacrifice the representational power of the short-term feature extractor by either freezing the backbone or using a small spatial video resolution. This issue becomes even worse with the recent video transformer models, many of which have quadratic memory complexity. To address these issues, we propose TALLFormer, a memory-efficient and end-to-end trainable Temporal Action Localization Transformer with Long-term memory. Our long-term memory mechanism eliminates the need for processing hundreds of redundant video frames during each training iteration, thus, significantly reducing the GPU memory consumption and training time. These efficiency savings allow us (i) to use a powerful video transformer feature extractor without freezing the backbone or reducing the spatial video resolution, while (ii) also maintaining long-range temporal boundary localization capability. With only RGB frames as input and no external action recognition classifier, TALLFormer outperforms previous state-of-the-arts by a large margin, achieving an average mAP of 59.1% on THUMOS14 and 35.6% on ActivityNet-1.3. The code is public available: https://github.com/klauscc/TALLFormer.","url_abs":"https://arxiv.org/abs/2204.01680v2","url_pdf":"https://arxiv.org/pdf/2204.01680v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tallformer-temporal-action-localization-with","repo_url":"https://github.com/klauscc/tallformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"action-localization","task_name":"Action Localization"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.01680","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.01680"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/klauscc/tallformer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"913d82af53065418","entry":"accuracy","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/multisegment_loss.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/multisegment_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"913d82af53065418"}},{"code_sha256_prefix":"a20b44c60cb60cfe","entry":"annos_transform","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/anet_dataset.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/anet_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a20b44c60cb60cfe"}},{"code_sha256_prefix":"9d5a412547d9a470","entry":"annos_transform","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/common/anet_dataset.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/common/anet_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d5a412547d9a470"}},{"code_sha256_prefix":"6d1c803e1058ac52","entry":"calc_ioa","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/multisegment_loss.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/multisegment_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6d1c803e1058ac52"}},{"code_sha256_prefix":"817d99d2d71a835d","entry":"dict_from_file","repo":"klauscc/tallformer","repo_kind":"official","path":"vedacore/fileio/parse.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/vedacore/fileio/parse.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"817d99d2d71a835d"}},{"code_sha256_prefix":"9755b4295b3da11f","entry":"get_video_info","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/anet_dataset.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/anet_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9755b4295b3da11f"}},{"code_sha256_prefix":"e1f7c9eb71128188","entry":"iou_loss","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/multisegment_loss.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/multisegment_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e1f7c9eb71128188"}},{"code_sha256_prefix":"dfe6fd88aa6076ef","entry":"list_from_file","repo":"klauscc/tallformer","repo_kind":"official","path":"vedacore/fileio/parse.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/vedacore/fileio/parse.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfe6fd88aa6076ef"}},{"code_sha256_prefix":"f59d6fac89230aac","entry":"load_json","repo":"klauscc/tallformer","repo_kind":"official","path":"AFSD/anet_video_cls/anet_dataset.py","file_url":"https://github.com/klauscc/tallformer/blob/HEAD/AFSD/anet_video_cls/anet_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f59d6fac89230aac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}