{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/wavemix-resource-efficient-token-mixing-for","title":"WaveMix: Resource-efficient Token Mixing for Images","arxiv_id":"2203.03689","date":"2022-03-07","proceeding":null,"authors":["Pranav Jeevan","Amit Sethi"],"abstract":"Although certain vision transformer (ViT) and CNN architectures generalize well on vision tasks, it is often impractical to use them on green, edge, or desktop computing due to their computational requirements for training and even testing. We present WaveMix as an alternative neural architecture that uses a multi-scale 2D discrete wavelet transform (DWT) for spatial token mixing. Unlike ViTs, WaveMix neither unrolls the image nor requires self-attention of quadratic complexity. Additionally, DWT introduces another inductive bias -- besides convolutional filtering -- to utilize the 2D structure of an image to improve generalization. The multi-scale nature of the DWT also reduces the requirement for a deeper architecture compared to the CNNs, as the latter relies on pooling for partial spatial mixing. WaveMix models show generalization that is competitive with ViTs, CNNs, and token mixers on several datasets while requiring lower GPU RAM (training and testing), number of computations, and storage. WaveMix have achieved State-of-the-art (SOTA) results in EMNIST Byclass and EMNIST Balanced datasets.","url_abs":"https://arxiv.org/abs/2203.03689v1","url_pdf":"https://arxiv.org/pdf/2203.03689v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"wavemix-resource-efficient-token-mixing-for","repo_url":"https://github.com/pranavphoenix/WaveMix","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-mnist","task":"Image Classification","dataset":"MNIST","model":"WaveMix-128/7","rank_in_archive_order":17,"of":81,"metrics":{"Percentage error":"0.29"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.03689","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.03689"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pranavphoenix/WaveMix","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8ed51e89287a2345","entry":"mode_to_int","repo":"pranavphoenix/WaveMix","repo_kind":"official","path":"Image_Classification/WaveMix_Version2.py","file_url":"https://github.com/pranavphoenix/WaveMix/blob/HEAD/Image_Classification/WaveMix_Version2.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ed51e89287a2345"}},{"code_sha256_prefix":"c18931526ac3006c","entry":"reflect","repo":"pranavphoenix/WaveMix","repo_kind":"official","path":"Image_Classification/WaveMix_Version2.py","file_url":"https://github.com/pranavphoenix/WaveMix/blob/HEAD/Image_Classification/WaveMix_Version2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c18931526ac3006c"}},{"code_sha256_prefix":"61229493ece46198","entry":"reflect","repo":"pranavphoenix/WaveMix","repo_kind":"official","path":"Semantic_Segmentation/WaveMix.py","file_url":"https://github.com/pranavphoenix/WaveMix/blob/HEAD/Semantic_Segmentation/WaveMix.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61229493ece46198"}},{"code_sha256_prefix":"cbba9f74dd6f7cce","entry":"sfb1d","repo":"pranavphoenix/WaveMix","repo_kind":"official","path":"Image_Classification/WaveMix_Version2.py","file_url":"https://github.com/pranavphoenix/WaveMix/blob/HEAD/Image_Classification/WaveMix_Version2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbba9f74dd6f7cce"}},{"code_sha256_prefix":"63548ddcf2956599","entry":"sfb1d","repo":"pranavphoenix/WaveMix","repo_kind":"official","path":"Semantic_Segmentation/WaveMix.py","file_url":"https://github.com/pranavphoenix/WaveMix/blob/HEAD/Semantic_Segmentation/WaveMix.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63548ddcf2956599"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}