Methods › Computer Vision › Image Model Blocks › Axial Attention
Axial Attention
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Axial Attention is a simple generalization of self-attention that naturally aligns with the multiple dimensions of the tensors in both the encoding and the decoding settings. It was first proposed in CCNet [1] named as criss-cross attention, which harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. Ho et al [2] extents CCNet to process multi-dimensional data. The proposed structure of the layers allows for the vast majority of the context to be computed in parallel during decoding without introducing any independence assumptions. It serves as the basic building block for developing self-attention-based autoregressive models for high-dimensional data tensors, e.g., Axial Transformers. It has been applied in AlphaFold [3] for interpreting protein sequences.
[1] Zilong Huang, Xinggang Wang, Lichao Huang, Chang Huang, Yunchao Wei, Wenyu Liu. CCNet: Criss-Cross Attention for Semantic Segmentation. ICCV, 2019.
[2] Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, Tim Salimans. arXiv:1912.12180
[3] Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, Tunyasuvunakool K, Bates R, Žídek A, Potapenko A, Bridgland A. Highly accurate protein structure prediction with AlphaFold. Nature. 2021 Jul 15:1-1.
Papers archive 2025-07-28
30 shown of 59, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing 23 May 2025 · 1 repository · arXiv:2505.17552
-
UNet with Axial Transformer : A Neural Weather Model for Precipitation Nowcasting 28 Apr 2025 · 1 repository · arXiv:2504.19408
-
MATEY: multiscale adaptive foundation models for spatiotemporal physical systems 29 Dec 2024 · 0 repositories · arXiv:2412.20601
-
Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI 16 Dec 2024 · 0 repositories · arXiv:2412.11849
-
ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis 3 Nov 2024 · 0 repositories · arXiv:2411.01564
-
Axial Attention Transformer Networks: A New Frontier in Breast Cancer Detection 18 Sep 2024 · 0 repositories · arXiv:2409.12347
-
EFCNet: Every Feature Counts for Small Medical Object Segmentation 26 Jun 2024 · 0 repositories · arXiv:2406.18201
-
Progressive Frequency-Aware Network for Laparoscopic Image Desmoking 19 Dec 2023 · 1 repository · arXiv:2312.12023
-
MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention 14 Dec 2023 · 1 repository · arXiv:2312.08866
-
Integrated Image and Location Analysis for Wound Classification: A Deep Learning Approach 23 Aug 2023 · 1 repository · arXiv:2308.11877
-
Scalable Deep Learning for RNA Secondary Structure Prediction 14 Jul 2023 · 1 repository · arXiv:2307.10073Syntology ran 6 of 6 samples · 0 unverified
-
Axial-LOB: High-Frequency Trading with Axial Attention 4 Dec 2022 · 1 repository · arXiv:2212.01807
-
Revisiting Image Pyramid Structure for High Resolution Salient Object Detection 20 Sep 2022 · 3 repositories · arXiv:2209.09475Syntology ran 3 of 11 samples · 8 unverified
-
GEM-2: Next Generation Molecular Property Prediction Network by Modeling Full-range Many-body Interactions 11 Aug 2022 · 1 repository · arXiv:2208.05863
-
CANet: Channel Extending and Axial Attention Catching Network for Multi-structure Kidney Segmentation 10 Aug 2022 · 0 repositories · arXiv:2208.05241
-
CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers 5 Jul 2022 · 2 repositories · arXiv:2207.02202Syntology ran 0 of 3 samples · 3 unverified
-
DAtRNet: Disentangling Fashion Attribute Embedding for Substitute Item Retrieval 1 Jun 2022 · 0 repositories
-
Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation 21 Mar 2022 · 1 repository · arXiv:2203.10900
-
Extending nn-UNet for brain tumor segmentation 9 Dec 2021 · 1 repository · arXiv:2112.04653
-
Modelling and Analysis of Magnetic Fields from Skeletal Muscle for Valuable Physiological Measurements 5 Apr 2021 · 0 repositories · arXiv:2104.02036
-
Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples 29 Mar 2021 · 0 repositories · arXiv:2103.15795
-
Motion Estimation for Optical Coherence Elastography Using Signal Phase and Intensity 19 Mar 2021 · 0 repositories · arXiv:2103.10784
-
Missing Cone Artifacts Removal in ODT using Unsupervised Deep Learning in Projection Domain 16 Mar 2021 · 0 repositories · arXiv:2103.09022
-
A learning-based view extrapolation method for axial super-resolution 11 Mar 2021 · 0 repositories · arXiv:2103.06510
-
Deep learning can differentiate IDH-mutant from IDH-wild type GBM 24 Feb 2021 · 0 repositories · arXiv:2102.13205
-
Axial Residual Networks for CycleGAN-based Voice Conversion 16 Feb 2021 · 0 repositories · arXiv:2102.08075
-
Colorization Transformer 8 Feb 2021 · 2 repositories · arXiv:2102.04432
-
Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation 19 Jan 2021 · 1 repository · arXiv:2101.07434
-
Unsupervised Domain Adaptation from Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks 19 Jan 2021 · 1 repository · arXiv:2101.07653
-
VideoGen: Generative Modeling of Videos using VQ-VAE and Transformers 1 Jan 2021 · 0 repositories
Tasks archive 2025-07-28
20 shown of 91 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Segmentation | 11 |
| Semantic Segmentation | 7 |
| Deep Learning | 5 |
| Generative Adversarial Network | 4 |
| Image Segmentation | 4 |
| Computational Efficiency | 3 |
| Decoder | 3 |
| Image Classification | 3 |
| Medical Image Segmentation | 3 |
| Object Detection | 3 |
| SSIM | 3 |
| Tumor Segmentation | 3 |
| image-classification | 3 |
| 3D Object Detection | 2 |
| Brain Tumor Segmentation | 2 |
| Diagnostic | 2 |
| Image Reconstruction | 2 |
| Motion Estimation | 2 |
| Position | 2 |
| Relation | 2 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections