{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transnext-robust-foveal-visual-perception-for","title":"TransNeXt: Robust Foveal Visual Perception for Vision Transformers","arxiv_id":"2311.17132","date":"2023-11-28","proceeding":"CVPR 2024 1","authors":["Dai Shi"],"abstract":"Due to the depth degradation effect in residual connections, many efficient Vision Transformers models that rely on stacking layers for information exchange often fail to form sufficient information mixing, leading to unnatural visual perception. To address this issue, in this paper, we propose Aggregated Attention, a biomimetic design-based token mixer that simulates biological foveal vision and continuous eye movement while enabling each token on the feature map to have a global perception. Furthermore, we incorporate learnable tokens that interact with conventional queries and keys, which further diversifies the generation of affinity matrices beyond merely relying on the similarity between queries and keys. Our approach does not rely on stacking for information exchange, thus effectively avoiding depth degradation and achieving natural visual perception. Additionally, we propose Convolutional GLU, a channel mixer that bridges the gap between GLU and SE mechanism, which empowers each token to have channel attention based on its nearest neighbor image features, enhancing local modeling capability and model robustness. We combine aggregated attention and convolutional GLU to create a new visual backbone called TransNeXt. Extensive experiments demonstrate that our TransNeXt achieves state-of-the-art performance across multiple model sizes. At a resolution of $224^2$, TransNeXt-Tiny attains an ImageNet accuracy of 84.0%, surpassing ConvNeXt-B with 69% fewer parameters. Our TransNeXt-Base achieves an ImageNet accuracy of 86.2% and an ImageNet-A accuracy of 61.6% at a resolution of $384^2$, a COCO object detection mAP of 57.1, and an ADE20K semantic segmentation mIoU of 54.7.","url_abs":"https://arxiv.org/abs/2311.17132v3","url_pdf":"https://arxiv.org/pdf/2311.17132v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transnext-robust-foveal-visual-perception-for","repo_url":"https://github.com/daishiresearch/transnext","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"transnext-robust-foveal-visual-perception-for","repo_url":"https://github.com/Westlake-AI/openmixup","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transnext-robust-foveal-visual-perception-for","repo_url":"https://github.com/chenller/mmseg-extension","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transnext-robust-foveal-visual-perception-for","repo_url":"https://gitlab.com/birder/birder","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/domain-generalization-on-imagenet-a","task":"Domain Generalization","dataset":"ImageNet-A","model":"TransNeXt-Base (IN-1K supervised, 384)","rank_in_archive_order":15,"of":39,"metrics":{"Number of params":"89.7M","Top-1 accuracy %":"61.6"},"uses_additional_data":false},{"leaderboard":"/sota/domain-generalization-on-imagenet-a","task":"Domain Generalization","dataset":"ImageNet-A","model":"TransNeXt-Small (IN-1K supervised, 384)","rank_in_archive_order":16,"of":39,"metrics":{"Number of params":"49.7M","Top-1 accuracy %":"58.3"},"uses_additional_data":false},{"leaderboard":"/sota/domain-generalization-on-imagenet-a","task":"Domain Generalization","dataset":"ImageNet-A","model":"TransNeXt-Base (IN-1K supervised, 224)","rank_in_archive_order":19,"of":39,"metrics":{"Number of params":"89.7M","Top-1 accuracy %":"50.6"},"uses_additional_data":false},{"leaderboard":"/sota/domain-generalization-on-imagenet-a","task":"Domain Generalization","dataset":"ImageNet-A","model":"TransNeXt-Small (IN-1K supervised, 224)","rank_in_archive_order":21,"of":39,"metrics":{"Number of params":"49.7M","Top-1 accuracy %":"47.1"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"TransNeXt-Base (IN-1K supervised, 384)","rank_in_archive_order":169,"of":1060,"metrics":{"GFLOPs":"56.3","Number of params":"89.7M","Top 1 Accuracy":"86.2%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"TransNeXt-Small (IN-1K supervised, 384)","rank_in_archive_order":179,"of":1060,"metrics":{"GFLOPs":"32.1","Number of params":"49.7M","Top 1 Accuracy":"86.0%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"TransNeXt-Small (IN-1K supervised, 224)","rank_in_archive_order":295,"of":1060,"metrics":{"GFLOPs":"10.3","Number of params":"49.7M","Top 1 Accuracy":"84.7%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"TransNeXt-Tiny (IN-1K supervised, 224)","rank_in_archive_order":365,"of":1060,"metrics":{"GFLOPs":"5.7","Number of params":"28.2M","Top 1 Accuracy":"84.0%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"TransNeXt-Micro (IN-1K supervised, 224)","rank_in_archive_order":528,"of":1060,"metrics":{"GFLOPs":"2.7","Number of params":"12.8M","Top 1 Accuracy":"82.5%"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"TransNeXt-Base (IN-1K pretrain, DINO 1x)","rank_in_archive_order":42,"of":220,"metrics":{"box AP":"57.1"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"TransNeXt-Small (IN-1K pretrain, DINO 1x)","rank_in_archive_order":44,"of":220,"metrics":{"box AP":"56.6"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"TransNeXt-Tiny (IN-1K pretrain, DINO 1x)","rank_in_archive_order":48,"of":220,"metrics":{"box AP":"55.7"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"TransNeXt-Base (IN-1K pretrain, Mask2Former, 512)","rank_in_archive_order":55,"of":235,"metrics":{"Params (M)":"109","Validation mIoU":"54.7"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"TransNeXt-Small (IN-1K pretrain, Mask2Former, 512)","rank_in_archive_order":68,"of":235,"metrics":{"Params (M)":"69","Validation mIoU":"54.1"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"TransNeXt-Tiny (IN-1K pretrain, Mask2Former, 512)","rank_in_archive_order":82,"of":235,"metrics":{"Params (M)":"47.5","Validation mIoU":"53.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.17132","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.17132"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Westlake-AI/openmixup","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/daishiresearch/transnext","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://gitlab.com/birder/birder","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chenller/mmseg-extension","reach":null}],"summary":{"ran_fixture":1,"ran":2,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"af9e1bafd1dd18f3","entry":"get_relative_position_cpb","repo":"chenller/mmseg-extension","repo_kind":"listed","path":"mmsegext/models/backbones/transnext_cuda.py","file_url":"https://github.com/chenller/mmseg-extension/blob/HEAD/mmsegext/models/backbones/transnext_cuda.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af9e1bafd1dd18f3"}},{"code_sha256_prefix":"f67dfbdd557f5e3f","entry":"get_relative_position_cpb","repo":"daishiresearch/transnext","repo_kind":"official","path":"classification/transnext.py","file_url":"https://github.com/daishiresearch/transnext/blob/HEAD/classification/transnext.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f67dfbdd557f5e3f"}},{"code_sha256_prefix":"e076f3ca82515c34","entry":"get_seqlen_and_mask","repo":"chenller/mmseg-extension","repo_kind":"listed","path":"mmsegext/models/backbones/transnext_native.py","file_url":"https://github.com/chenller/mmseg-extension/blob/HEAD/mmsegext/models/backbones/transnext_native.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e076f3ca82515c34"}},{"code_sha256_prefix":"973ff8b81a13f50b","entry":"get_seqlen_scale","repo":"chenller/mmseg-extension","repo_kind":"listed","path":"mmsegext/models/backbones/transnext_cuda.py","file_url":"https://github.com/chenller/mmseg-extension/blob/HEAD/mmsegext/models/backbones/transnext_cuda.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"973ff8b81a13f50b"}},{"code_sha256_prefix":"df3275abf9c53939","entry":"is_installed","repo":"daishiresearch/transnext","repo_kind":"official","path":"classification/transnext.py","file_url":"https://github.com/daishiresearch/transnext/blob/HEAD/classification/transnext.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"df3275abf9c53939"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}