{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/acceltran-a-sparsity-aware-accelerator-for","title":"AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers","arxiv_id":"2302.14705","date":"2023-02-28","proceeding":null,"authors":["Shikhar Tuli","Niraj K. Jha"],"abstract":"Self-attention-based transformer models have achieved tremendous success in the domain of natural language processing. Despite their efficacy, accelerating the transformer is challenging due to its quadratic computational complexity and large activation sizes. Existing transformer accelerators attempt to prune its tokens to reduce memory access, albeit with high compute overheads. Moreover, previous works directly operate on large matrices involved in the attention operation, which limits hardware utilization. In order to address these challenges, this work proposes a novel dynamic inference scheme, DynaTran, which prunes activations at runtime with low overhead, substantially reducing the number of ineffectual operations. This improves the throughput of transformer inference. We further propose tiling the matrices in transformer operations along with diverse dataflows to improve data reuse, thus enabling higher energy efficiency. To effectively implement these methods, we propose AccelTran, a novel accelerator architecture for transformers. Extensive experiments with different models and benchmarks demonstrate that DynaTran achieves higher accuracy than the state-of-the-art top-k hardware-aware pruning strategy while attaining up to 1.2$\\times$ higher sparsity. One of our proposed accelerators, AccelTran-Edge, achieves 330K$\\times$ higher throughput with 93K$\\times$ lower energy requirement when compared to a Raspberry Pi device. On the other hand, AccelTran-Server achieves 5.73$\\times$ higher throughput and 3.69$\\times$ lower energy consumption compared to the state-of-the-art transformer co-processor, Energon. The simulation source code is available at https://github.com/jha-lab/acceltran.","url_abs":"https://arxiv.org/abs/2302.14705v2","url_pdf":"https://arxiv.org/pdf/2302.14705v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"acceltran-a-sparsity-aware-accelerator-for","repo_url":"https://github.com/jha-lab/acceltran","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2302.14705","atlas_url":"https://app.syntology.ai/?focus=2302.14705","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.14705"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jha-lab/acceltran","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5e454a59cd8c0217","entry":"get_op_list","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/simulator.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/simulator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5e454a59cd8c0217"}},{"code_sha256_prefix":"a557c22ab2527687","entry":"get_ops_done","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/run_dataflow.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/run_dataflow.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a557c22ab2527687"}},{"code_sha256_prefix":"43e5c13ecbd9f06d","entry":"get_tiled_ops","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/dict2ops.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/dict2ops.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"43e5c13ecbd9f06d"}},{"code_sha256_prefix":"790139f3f26c1fad","entry":"get_tokenizer_args","repo":"jha-lab/acceltran","repo_kind":"official","path":"pruning/run_evaluation.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/pruning/run_evaluation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"790139f3f26c1fad"}},{"code_sha256_prefix":"e2105743f5a5b342","entry":"get_training_args","repo":"jha-lab/acceltran","repo_kind":"official","path":"pruning/run_evaluation.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/pruning/run_evaluation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e2105743f5a5b342"}},{"code_sha256_prefix":"e21722b02b27cae9","entry":"update_op_idx","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/simulator.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/simulator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e21722b02b27cae9"}},{"code_sha256_prefix":"54725a6fca4e8125","entry":"get_last_compute_op","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/simulator.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/simulator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"54725a6fca4e8125"}},{"code_sha256_prefix":"92d7fe452f29b732","entry":"get_ops","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/dict2ops.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/dict2ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"92d7fe452f29b732"}},{"code_sha256_prefix":"8532da00c39e5882","entry":"main","repo":"jha-lab/acceltran","repo_kind":"official","path":"simulator/src/dict2ops.py","file_url":"https://github.com/jha-lab/acceltran/blob/HEAD/simulator/src/dict2ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8532da00c39e5882"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}