{"url":"/method/sca","slug":"sca","name":"SCA","full_name":"Semantic Cross Attention","full_name_withheld":false,"description_markdown":"Semantic Cross Attention (SCA) is based on cross attention, which we restrict with respect to a semantic mask.\r\n\r\nThe goal of SCA is two-fold depending on what is the query and what is the key. Either it allows to give the feature map information from a semantically restricted set of latents or, respectively, it allows a set of latents to retrieve information in a semantically restricted region of the feature map. \r\n\r\nSCA is defined as:  \r\n\r\n\\begin{equation}\r\n    \\text{SCA}(I_{1}, I_{2}, I_{3}) = \\sigma\\left(\\frac{QK^T\\odot I_{3} +\\tau \\left(1-I_{3}\\right)}{\\sqrt{d_{in}}}\\right)V \\quad ,\r\n\\end{equation}\r\n\r\nwhere $I_{1},I_{2},I_{3}$ the inputs, with $I_{1}$ attending $I_{2}$, and $I_{3}$ the mask that forces tokens from $I_1$ to attend only specific tokens from $I_2$. The attention values requiring masking are filled with $-\\infty$ before the softmax. (In practice $\\tau{=}-10^9$),  $Q {=} W_QI_{1}$, $K {=} W_KI_{2}$ and $V {=} W_VI_{2}$ the queries, keys and values, and $d_{in}$ the internal attention dimension. $\\sigma(.)$ is the softmax operation.\r\n\r\nLet $X\\in\\mathbb{R}^{n\\times C}$ be the feature map with n the number of pixels, and C the number of channels. Let $Z\\in\\mathbb{R}^{m\\times d}$ be a set of $m$ latents of dimension $d$ and $s$ the number of semantic labels. Each semantic label is attributed $k$ latents, such that $m=k\\times s$. Each semantic label mask is assigned $k$ copies in $S{\\in}\\{0;1\\}^{n \\times m}$. \r\n\r\nWe can differentiate 3 types of SCA:\r\n\r\n(a) SCA with pixels $X$ attending latents $Z$: $\\text{SCA}(X, Z, S)$, where $W_{Q} {\\in} \\mathbb{R}^{n\\times d_{in}}$ and $W_{K}, W_{V} {\\in} \\mathbb{R}^{m\\times d_{in}}$.\r\nThe idea is to force the pixels from a semantic region to attend latents that are associated with the same label. \r\n\r\n(b) SCA with latents $Z$ attending pixels $X$: $\\text{SCA}(Z, X, S)$, where $W_{Q}{\\in} \\mathbb{R}^{m\\times d_{in}}$, $W_{K}, W_{V} {\\in} \\mathbb{R}^{n\\times d_{in}}$. \r\nThe idea is to semantically mask attention values to enforce latents to attend semantically corresponding pixels.\r\n\r\n(c) SCA with latents $Z$ attending themselves: $\\text{SCA}(Z, Z, M)$, where $W_{Q}, W_{K}, W_{V} {\\in} \\mathbb{R}^{n\\times d_{in}}$. We denote $M\\in\\mathbb{N}^{m\\times m}$ this mask, with $M_{\\text{latents}}(i,j) {=} 1$ if the semantic label of latent $i$ is the same as the one of latent $j$; $0$ otherwise.\r\nThe idea is to let the latents only attend latents that share the same semantic label.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/2210.04883v1","title":"SCAM! Transferring humans between images with Semantic Cross Attention Modulation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Attention Modules","url":"/methods/category/attention-modules","pwc_aliases":[]}],"n_papers_tagged":47,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"UAV-Based Remote Sensing of Soil Moisture Across Diverse Land Covers: Validation and Bayesian Uncertainty Characterization","date":"2025-06-05","arxiv_id":"2506.05572","n_code_links":0,"syntology":null},{"paper":null,"title":"Pinching-Antenna Systems (PASS)-enabled Secure Wireless Communications","date":"2025-04-18","arxiv_id":"2504.13670","n_code_links":0,"syntology":null},{"paper":null,"title":"Probabilistic Stability Guarantees for Feature Attributions","date":"2025-04-18","arxiv_id":"2504.13787","n_code_links":0,"syntology":null},{"paper":null,"title":"Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator","date":"2025-02-05","arxiv_id":"2502.02972","n_code_links":0,"syntology":null},{"paper":null,"title":"It's Not Just a Phase: On Investigating Phase Transitions in Deep Learning-based Side-channel Analysis","date":"2025-02-01","arxiv_id":"2502.00384","n_code_links":0,"syntology":null},{"paper":null,"title":"Movable Antenna-Assisted Integrated Sensing and Communication Systems","date":"2025-01-02","arxiv_id":"2501.01217","n_code_links":0,"syntology":null},{"paper":null,"title":"Make Shuffling Great Again: A Side-Channel Resistant Fisher-Yates Algorithm for Protecting Neural Networks","date":"2025-01-01","arxiv_id":"2501.00798","n_code_links":0,"syntology":null},{"paper":null,"title":"High-Throughput Detection of Risk Factors to Sudden Cardiac Arrest in Youth Athletes: A Smartwatch-Based Screening Platform","date":"2024-12-02","arxiv_id":"2412.12118","n_code_links":0,"syntology":null},{"paper":null,"title":"Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation","date":"2024-12-02","arxiv_id":"2412.01316","n_code_links":0,"syntology":null},{"paper":"/paper/fonts-text-rendering-with-typography-and","title":"FonTS: Text Rendering with Typography and Style Controls","date":"2024-11-28","arxiv_id":"2412.00136","n_code_links":1,"syntology":null},{"paper":null,"title":"Multi-task Gaze Estimation Via Unidirectional Convolution","date":"2024-11-27","arxiv_id":"2411.18061","n_code_links":0,"syntology":null},{"paper":null,"title":"Power Source Allocation for RIS-aided Integrating Sensing, Communication, and Power Transfer Systems Based on NOMA","date":"2024-11-01","arxiv_id":"2411.00334","n_code_links":0,"syntology":null},{"paper":"/paper/semantic-component-analysis-discovering","title":"Semantic Component Analysis: Discovering Patterns in Short Texts Beyond Topics","date":"2024-10-28","arxiv_id":"2410.21054","n_code_links":1,"syntology":null},{"paper":null,"title":"Mutual Coupling-Aware Channel Estimation and Beamforming for RIS-Assisted Communications","date":"2024-10-05","arxiv_id":"2410.04110","n_code_links":0,"syntology":null},{"paper":"/paper/sca-highly-efficient-semantic-consistent","title":"SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion","date":"2024-10-03","arxiv_id":"2410.02240","n_code_links":1,"syntology":null},{"paper":null,"title":"Optimizing Multi-User Uplink Cooperative Rate-Splitting Multiple Access: Efficient User Pairing and Resource Allocation","date":"2024-09-03","arxiv_id":"2409.02276","n_code_links":0,"syntology":null},{"paper":null,"title":"FAS vs. ARIS: Which Is More Important for FAS-ARIS Communication Systems?","date":"2024-08-17","arxiv_id":"2408.09067","n_code_links":0,"syntology":null},{"paper":null,"title":"Fluid-Antenna Enhanced ISAC: Joint Antenna Positioning and Dual-Functional Beamforming Design under Perfect and Imperfect CSI","date":"2024-07-25","arxiv_id":"2407.18988","n_code_links":0,"syntology":null},{"paper":null,"title":"Side-Channel Analysis of OpenVINO-based Neural Network Models","date":"2024-07-23","arxiv_id":"2407.16467","n_code_links":0,"syntology":null},{"paper":null,"title":"Multi-Functional Beamforming Design for Integrated Sensing, Communication, and Computation","date":"2024-07-01","arxiv_id":"2407.01006","n_code_links":0,"syntology":null},{"paper":null,"title":"Brain Tumor Classification using Vision Transformer with Selective Cross-Attention Mechanism and Feature Calibration","date":"2024-06-25","arxiv_id":"2406.17670","n_code_links":0,"syntology":null},{"paper":"/paper/depth-linear-discrimination-oriented-feature","title":"Depth linear discrimination-oriented feature selection method based on adaptive sine cosine algorithm for software defect prediction","date":"2024-05-20","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Impedance vs. Power Side-channel Vulnerabilities: A Comparative Study","date":"2024-05-10","arxiv_id":"2405.06242","n_code_links":0,"syntology":null},{"paper":null,"title":"Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models","date":"2024-05-09","arxiv_id":"2405.05990","n_code_links":0,"syntology":null},{"paper":null,"title":"Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation","date":"2024-03-18","arxiv_id":"2403.11960","n_code_links":0,"syntology":null},{"paper":null,"title":"SNOW-SCA: ML-assisted Side-Channel Attack on SNOW-V","date":"2024-03-13","arxiv_id":"2403.08267","n_code_links":0,"syntology":null},{"paper":null,"title":"Precise Extraction of Deep Learning Models via Side-Channel Attacks on Edge/Endpoint Devices","date":"2024-03-05","arxiv_id":"2403.02870","n_code_links":0,"syntology":null},{"paper":null,"title":"A Review and Comparison of AI Enhanced Side Channel Analysis","date":"2024-02-03","arxiv_id":"2402.02299","n_code_links":0,"syntology":null},{"paper":null,"title":"On the Sum Secrecy Rate Maximisation for Wireless Vehicular Networks","date":"2024-01-31","arxiv_id":"2401.17522","n_code_links":0,"syntology":null},{"paper":"/paper/energy-efficient-power-allocation-in-cell","title":"Energy-Efficient Power Allocation in Cell-Free Massive MIMO via Graph Neural Networks","date":"2024-01-25","arxiv_id":"2401.14281","n_code_links":1,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/side-channel-analysis","name":"Side Channel Analysis","papers":7},{"task":"/task/integrated-sensing-and-communication","name":"Integrated sensing and communication","papers":4},{"task":"/task/deep-learning","name":"Deep Learning","papers":3},{"task":"/task/diversity","name":"Diversity","papers":3},{"task":"/task/isac","name":"ISAC","papers":3},{"task":"/task/image-generation","name":"Image Generation","papers":2},{"task":null,"name":"Position","papers":2},{"task":"/task/adversarial-attack","name":"Adversarial Attack","papers":1},{"task":"/task/adversarial-robustness","name":"Adversarial Robustness","papers":1},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":1},{"task":"/task/bayesian-inference","name":"Bayesian Inference","papers":1},{"task":"/task/brain-tumor-classification","name":"Brain Tumor Classification","papers":1},{"task":"/task/classification-1","name":"Classification","papers":1},{"task":"/task/conditional-image-generation","name":"Conditional Image Generation","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/federated-learning","name":"Federated Learning","papers":1},{"task":"/task/gaussian-processes","name":"Gaussian Processes","papers":1},{"task":"/task/gaze-estimation","name":"Gaze Estimation","papers":1},{"task":"/task/graph-neural-network","name":"Graph Neural Network","papers":1}],"tasks_shown":20,"n_tasks":47,"usage_by_year":[{"year":"2022","papers":4},{"year":"2023","papers":11},{"year":"2024","papers":25},{"year":"2025","papers":7}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sca"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}