{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mcuformer-deploying-vision-tranformers-on-1","title":"MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory","arxiv_id":"2310.16898","date":"2023-10-25","proceeding":"NeurIPS 2023 11","authors":["Yinan Liang","Ziwei Wang","Xiuwei Xu","Yansong Tang","Jie zhou","Jiwen Lu"],"abstract":"Due to the high price and heavy energy consumption of GPUs, deploying deep models on IoT devices such as microcontrollers makes significant contributions for ecological AI. Conventional methods successfully enable convolutional neural network inference of high resolution images on microcontrollers, while the framework for vision transformers that achieve the state-of-the-art performance in many vision applications still remains unexplored. In this paper, we propose a hardware-algorithm co-optimizations method called MCUFormer to deploy vision transformers on microcontrollers with extremely limited memory, where we jointly design transformer architecture and construct the inference operator library to fit the memory resource constraint. More specifically, we generalize the one-shot network architecture search (NAS) to discover the optimal architecture with highest task performance given the memory budget from the microcontrollers, where we enlarge the existing search space of vision transformers by considering the low-rank decomposition dimensions and patch resolution for memory reduction. For the construction of the inference operator library of vision transformers, we schedule the memory buffer during inference through operator integration, patch embedding decomposition, and token overwriting, allowing the memory buffer to be fully utilized to adapt to the forward pass of the vision transformer. Experimental results demonstrate that our MCUFormer achieves 73.62\\% top-1 accuracy on ImageNet for image classification with 320KB memory on STM32F746 microcontroller. Code is available at https://github.com/liangyn22/MCUFormer.","url_abs":"https://arxiv.org/abs/2310.16898v3","url_pdf":"https://arxiv.org/pdf/2310.16898v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mcuformer-deploying-vision-tranformers-on-1","repo_url":"https://github.com/liangyn22/mcuformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.16898","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.16898"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/liangyn22/MCUFormer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liangyn22/mcuformer","reach":null}],"summary":{"ran":4,"ran_draft_wrong":4,"ran_honours":2,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"46ae2d4f3fc0bdd6","entry":"LinearSuper","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"46ae2d4f3fc0bdd6"}},{"code_sha256_prefix":"a137a6efb264aa07","entry":"PatchembedSuper","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a137a6efb264aa07"}},{"code_sha256_prefix":"701ac4868b3a9368","entry":"RelativePosition2D_super","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"701ac4868b3a9368"}},{"code_sha256_prefix":"e2c1a7dffcdbe918","entry":"_ntuple","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2c1a7dffcdbe918"}},{"code_sha256_prefix":"72a860a9aff50105","entry":"calc_dropout","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72a860a9aff50105"}},{"code_sha256_prefix":"b328ce4411979624","entry":"decode_cand_tuple","repo":"liangyn22/MCUFormer","repo_kind":"official","path":"evolution.py","file_url":"https://github.com/liangyn22/MCUFormer/blob/HEAD/evolution.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b328ce4411979624"}},{"code_sha256_prefix":"08a5be1ffa0676e3","entry":"gelu","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"08a5be1ffa0676e3"}},{"code_sha256_prefix":"c356588a20cf831d","entry":"qkv_super","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c356588a20cf831d"}},{"code_sha256_prefix":"963bc33b77cf8148","entry":"sample_bias","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"963bc33b77cf8148"}},{"code_sha256_prefix":"1d3488a4b38ea75d","entry":"sample_weight","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1d3488a4b38ea75d"}},{"code_sha256_prefix":"765f54d24658a76a","entry":"AttentionSuper","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"765f54d24658a76a"}},{"code_sha256_prefix":"e4f3e6207e219ef8","entry":"LayerNormSuper","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4f3e6207e219ef8"}},{"code_sha256_prefix":"a115d422c7fbcbe2","entry":"TransformerEncoderLayer","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a115d422c7fbcbe2"}},{"code_sha256_prefix":"09a88c3ccde67487","entry":"Vision_TransformerSuper","repo":"liangyn22/mcuformer","repo_kind":"official","path":"model/supernet_transformer.py","file_url":"https://github.com/liangyn22/mcuformer/blob/HEAD/model/supernet_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09a88c3ccde67487"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}