{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lookupffn-making-transformers-compute-lite","title":"LookupFFN: Making Transformers Compute-lite for CPU inference","arxiv_id":"2403.07221","date":"2024-03-12","proceeding":null,"authors":["Zhanpeng Zeng","Michael Davies","Pranav Pulijala","Karthikeyan Sankaralingam","Vikas Singh"],"abstract":"While GPU clusters are the de facto choice for training large deep neural network (DNN) models today, several reasons including ease of workflow, security and cost have led to efforts investigating whether CPUs may be viable for inference in routine use in many sectors of the industry. But the imbalance between the compute capabilities of GPUs and CPUs is huge. Motivated by these considerations, we study a module which is a workhorse within modern DNN architectures, GEMM based Feed Forward Networks (FFNs), and assess the extent to which it can be made compute- (or FLOP-) lite. Specifically, we propose an alternative formulation (we call it LookupFFN) to GEMM based FFNs inspired by the recent studies of using Locality Sensitive Hashing (LSH) to approximate FFNs. Our formulation recasts most essential operations as a memory look-up, leveraging the trade-off between the two resources on any platform: compute and memory (since CPUs offer it in abundance). For RoBERTa language model pretraining, our formulation achieves similar performance compared to GEMM based FFNs, while dramatically reducing the required FLOP. Our development is complemented with a detailed hardware profiling of strategies that will maximize efficiency -- not just on contemporary hardware but on products that will be offered in the near/medium term future. Code is avaiable at \\url{https://github.com/mlpen/LookupFFN}.","url_abs":"https://arxiv.org/abs/2403.07221v1","url_pdf":"https://arxiv.org/pdf/2403.07221v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lookupffn-making-transformers-compute-lite","repo_url":"https://github.com/mlpen/lookupffn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"roberta","method_name":"RoBERTa"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.07221","atlas_url":"https://app.syntology.ai/?focus=2403.07221","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.07221"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mlpen/lookupffn","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"661289c0e647ffa7","entry":"create_dataset","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/datasets_scripts/downstream/quality.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/datasets_scripts/downstream/quality.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"661289c0e647ffa7"}},{"code_sha256_prefix":"a52fce88045dcb83","entry":"filter_inputs","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a52fce88045dcb83"}},{"code_sha256_prefix":"8348dc66b62ce338","entry":"get_optimizer","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/base_model_module.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/base_model_module.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8348dc66b62ce338"}},{"code_sha256_prefix":"a00e41e74ad0c0e6","entry":"import_config","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/args.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/args.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a00e41e74ad0c0e6"}},{"code_sha256_prefix":"204b883a4c318423","entry":"import_from_string","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/args.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/args.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"204b883a4c318423"}},{"code_sha256_prefix":"68d026dc0e4a5dfc","entry":"write_arrow_file","repo":"mlpen/lookupffn","repo_kind":"official","path":"src/datasets_scripts/pretrain/jsonl_to_arrow.py","file_url":"https://github.com/mlpen/lookupffn/blob/HEAD/src/datasets_scripts/pretrain/jsonl_to_arrow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68d026dc0e4a5dfc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}