Methods › Computer Vision › Vision Transformers › XCiT
XCiT
Introduced by Alaaeldin El-Nouby et al. in XCiT: Cross-Covariance Image Transformers
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Cross-Covariance Image Transformers, or XCiT, is a type of vision transformer that aims to combine the accuracy of conventional transformers with the scalability of convolutional architectures.
The self-attention operation underlying transformers yields global interactions between all tokens, i.e. words or image patches, and enables flexible modelling of image data beyond the local interactions of convolutions. This flexibility, however, comes with a quadratic complexity in time and memory, hindering application to long sequences and high-resolution images. The authors propose a “transposed” version of self-attention called cross-covariance attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariances matrix between keys and queries.
Papers archive 2025-07-28
4 shown of 4, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models 14 Oct 2022 · 0 repositories · arXiv:2210.09223
-
SimA: Simple Softmax-free Attention for Vision Transformers 17 Jun 2022 · 1 repository · arXiv:2206.08898Syntology ran 0 of 3 samples · 3 unverified
-
PE-former: Pose Estimation Transformer 9 Dec 2021 · 1 repository · arXiv:2112.04981
-
XCiT: Cross-Covariance Image Transformers 17 Jun 2021 · 12 repositories · arXiv:2106.09681Syntology ran 3 of 14 samples · 11 unverified · 3 pointer-only (licence)
Tasks archive 2025-07-28
10 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections