Papers › Attentive Normalization
Attentive Normalization
Xilai Li, Wei Sun, Tianfu Wu
In state-of-the-art deep neural networks, both feature normalization and feature attention have become ubiquitous. % with significant performance improvement shown in a vast amount of tasks. They are usually studied as separate modules, however. In this paper, we propose a light-weight integration between the two schema and present Attentive Normalization (AN). Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging channel-wise feature attention. In experiments, we test the proposed AN using four representative neural architectures in the ImageNet-1000 classification benchmark and the MS-COCO 2017 object detection and instance segmentation benchmark. AN obtains consistent performance improvement for different neural architectures in both benchmarks with absolute increase of top-1 accuracy in ImageNet-1000 between 0.5\% and 2.7\%, and absolute increase up to 1.8\% and 2.2\% for bounding box and mask AP in MS-COCO respectively. We observe that the proposed AN provides a strong alternative to the widely used Squeeze-and-Excitation (SE) module. The source codes are publicly available at https://github.com/iVMCL/AOGNet-v2 (the ImageNet Classification Repo) and https://github.com/iVMCL/AttentiveNorm\_Detection (the MS-COCO Detection and Segmentation Repo).
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image Classification | ImageNet | AOGNet-40M-AN | GFLOPs | 7.51 | #602 of 1060 | Archive leaderboard | report |
| Image Classification | ImageNet | AOGNet-40M-AN | Top 1 Accuracy | 81.87% | #602 of 1060 | Archive leaderboard | report |
| Instance Segmentation | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | AP50 | 63.2 | #74 of 93 | Archive leaderboard | report |
| Instance Segmentation | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | AP75 | 43.3 | #74 of 93 | Archive leaderboard | report |
| Instance Segmentation | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | mask AP | 40.2 | #74 of 93 | Archive leaderboard | report |
| Object Detection | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | AP50 | 66.2 | #119 of 220 | Archive leaderboard | report |
| Object Detection | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | AP75 | 49.1 | #119 of 220 | Archive leaderboard | report |
| Object Detection | COCO minival | Mask R-CNN-FPN (AOGNet-40M) | box AP | 44.9 | #119 of 220 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections