</> Code with Papers by Syntology

Methods › General › Attention Modules › SCA

Semantic Cross Attention

SCA

47 papers tagged archive 2025-07-28

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Semantic Cross Attention (SCA) is based on cross attention, which we restrict with respect to a semantic mask.

The goal of SCA is two-fold depending on what is the query and what is the key. Either it allows to give the feature map information from a semantically restricted set of latents or, respectively, it allows a set of latents to retrieve information in a semantically restricted region of the feature map.

SCA is defined as:

SCA(I₁, I₂, I₃) = σ((QKᵀ⊙I₃ +τ(1-I₃))/(√(dᵢₙ)))V ,

where I₁,I₂,I₃ the inputs, with I₁ attending I₂, and I₃ the mask that forces tokens from I₁ to attend only specific tokens from I₂. The attention values requiring masking are filled with -∞ before the softmax. (In practice τ=-10⁹), Q = W_QI₁, K = W_KI₂ and V = W_VI₂ the queries, keys and values, and dᵢₙ the internal attention dimension. σ(.) is the softmax operation.

Let X∈ℝ^(n×C) be the feature map with n the number of pixels, and C the number of channels. Let Z∈ℝ^(m×d) be a set of m latents of dimension d and s the number of semantic labels. Each semantic label is attributed k latents, such that m=k×s. Each semantic label mask is assigned k copies in S∈{0;1}^(n ×m).

We can differentiate 3 types of SCA:

(a) SCA with pixels X attending latents Z: SCA(X, Z, S), where W_Q ∈ ℝ^(n×dᵢₙ) and W_K, W_V ∈ ℝ^(m×dᵢₙ). The idea is to force the pixels from a semantic region to attend latents that are associated with the same label.

(b) SCA with latents Z attending pixels X: SCA(Z, X, S), where W_Q∈ ℝ^(m×dᵢₙ), W_K, W_V ∈ ℝ^(n×dᵢₙ). The idea is to semantically mask attention values to enforce latents to attend semantically corresponding pixels.

(c) SCA with latents Z attending themselves: SCA(Z, Z, M), where W_Q, W_K, W_V ∈ ℝ^(n×dᵢₙ). We denote M∈ℕ^(m×m) this mask, with Mₗₐₜₑₙₜₛ(i,j) = 1 if the semantic label of latent i is the same as the one of latent j; $0$ otherwise. The idea is to let the latents only attend latents that share the same semantic label.

Source: SCAM! Transferring humans between images with Semantic...

Papers archive 2025-07-28

30 shown of 47, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 47 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Side Channel Analysis7
Integrated sensing and communication4
Deep Learning3
Diversity3
ISAC3
Image Generation2
Position2
Adversarial Attack1
Adversarial Robustness1
Autonomous Driving1
Bayesian Inference1
Brain Tumor Classification1
Classification1
Conditional Image Generation1
Decoder1
Denoising1
Federated Learning1
Gaussian Processes1
Gaze Estimation1
Graph Neural Network1

Usage over time archive 2025-07-28

Papers per year tagged with SCA: 2022 to 2025, peak 25 25 0 2022: 4 papers 2022 2023: 11 papers 2023 2024: 25 papers 2024 2025: 7 papers 2025
Papers per year the archive tags with this method, by the paper's archive date (47 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Attention Modules

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections