Methods › Computer Vision › Image Model Blocks › XCiT Layer

XCiT Layer

4 papers tagged archive 2025-07-28

Introduced by Alaaeldin El-Nouby et al. in XCiT: Cross-Covariance Image Transformers

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

An XCiT Layer is the main building block of the XCiT architecture which uses a cross-covariance attention operator as its principal operation. The XCiT layer consists of three main blocks, each preceded by LayerNorm and followed by a residual connection: (i) the core cross-covariance attention (XCA) operation, (ii) the local patch interaction (LPI) module, and (iii) a feed-forward network (FFN). By transposing the query-key interaction, the computational complexity of XCA is linear in the number of data elements N, rather than quadratic as in conventional self-attention.

Paper Source

Papers archive 2025-07-28

4 shown of 4, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models
14 Oct 2022 · 0 repositories · arXiv:2210.09223
SimA: Simple Softmax-free Attention for Vision Transformers
17 Jun 2022 · 1 repository · arXiv:2206.08898

Syntology ran 0 of 3 samples · 3 unverified
PE-former: Pose Estimation Transformer
9 Dec 2021 · 1 repository · arXiv:2112.04981
XCiT: Cross-Covariance Image Transformers
17 Jun 2021 · 12 repositories · arXiv:2106.09681

Syntology ran 3 of 14 samples · 11 unverified · 3 pointer-only (licence)

Tasks archive 2025-07-28

10 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

Task	Papers
Image Classification	3
image-classification	2
Decoder	1
Instance Segmentation	1
Object Detection	1
Pose Estimation	1
Quantization	1
Self-Supervised Image Classification	1
Semantic Segmentation	1
object-detection	1

Usage over time archive 2025-07-28

Papers per year the archive tags with this method, by the paper's archive date (4 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Image Model Blocks

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections