Papers › Rethinking Local Perception in Lightweight Vision Transformer

Rethinking Local Perception in Lightweight Vision Transformer

31 Mar 2023arXiv:2303.17803archive 2025-07-28

Qihang Fan, Huaibo Huang, Jiyang Guan, Ran He

Vision Transformers (ViTs) have been shown to be effective in various vision tasks. However, resizing them to a mobile-friendly size leads to significant performance degradation. Therefore, developing lightweight vision transformers has become a crucial area of research. This paper introduces CloFormer, a lightweight vision transformer that leverages context-aware local enhancement. CloFormer explores the relationship between globally shared weights often used in vanilla convolutional operators and token-specific context-aware weights appearing in attention, then proposes an effective and straightforward module to capture high-frequency local information. In CloFormer, we introduce AttnConv, a convolution operator in attention's style. The proposed AttnConv uses shared weights to aggregate local information and deploys carefully designed context-aware weights to enhance local features. The combination of the AttnConv and vanilla attention which uses pooling to reduce FLOPs in CloFormer enables the model to perceive high-frequency and low-frequency information. Extensive experiments were conducted in image classification, object detection, and semantic segmentation, demonstrating the superiority of CloFormer. The code is available at \url{https://github.com/qhfan/CloFormer}.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

qhfan/CloFormer officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Classification Object Detection Semantic Segmentation image-classification object-detection

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Classification	ImageNet	CloFormer-S	GFLOPs	2	#623 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-S	Number of params	12.3M	#623 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-S	Top 1 Accuracy	81.6%	#623 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XS	GFLOPs	1.1	#738 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XS	Number of params	7.2M	#738 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XS	Top 1 Accuracy	79.8%	#738 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XXS	GFLOPs	0.6	#892 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XXS	Number of params	4.2M	#892 of 1060	Archive leaderboard	report
Image Classification	ImageNet	CloFormer-XXS	Top 1 Accuracy	77%	#892 of 1060	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Attention Convolution Dense Connections Layer Normalization Linear Layer Multi-Head Attention Residual Connection Softmax Vision Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections