Methods › General › Attention Mechanisms › Set Transformer
Set Transformer
Introduced by Juho Lee et al. in Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set, models used to address them should be permutation invariant. We present an attention-based neural network module, the Set Transformer, specifically designed to model interactions among elements in the input set. The model consists of an encoder and a decoder, both of which rely on attention mechanisms. In an effort to reduce computational complexity, we introduce an attention scheme inspired by inducing point methods from sparse Gaussian process literature. It reduces the computation time of self-attention from quadratic to linear in the number of elements in the set. We show that our model is theoretically attractive and we evaluate it on a range of tasks, demonstrating the state-of-the-art performance compared to recent methods for set-structured data.
Papers archive 2025-07-28
20 shown of 20, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
To Bin or not to Bin: Alternative Representations of Mass Spectra 15 Feb 2025 · 0 repositories · arXiv:2502.10851
-
Advances in Set Function Learning: A Survey of Techniques and Applications 24 Jan 2025 · 0 repositories · arXiv:2501.14991
-
Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks 20 Jan 2025 · 0 repositories · arXiv:2501.11350
-
Multiset Transformer: Advancing Representation Learning in Persistence Diagrams 22 Nov 2024 · 1 repository · arXiv:2411.14662
-
Graph as Point Set 5 May 2024 · 1 repository · arXiv:2405.02795Syntology ran 20 of 27 samples · 7 unverified · 27 pointer-only (licence)
-
Associative Transformer 22 Sep 2023 · 1 repository · arXiv:2309.12862
-
Pointersect: Neural Rendering with Cloud-Ray Intersection 24 Apr 2023 · 0 repositories · arXiv:2304.12390
-
Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams 7 Mar 2023 · 1 repository · arXiv:2303.03856
-
Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets 23 Jun 2022 · 1 repository · arXiv:2206.11925Syntology ran 1 of 2 samples · 1 unverified
-
Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation 11 Apr 2022 · 0 repositories · arXiv:2204.04913
-
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds 19 Mar 2022 · 1 repository · arXiv:2203.10314Syntology ran 0 of 6 samples · 6 unverified
-
Automated Identification of Cell Populations in Flow Cytometry Data with Transformers 23 Aug 2021 · 1 repository · arXiv:2108.10072
-
You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks 24 Jun 2021 · 1 repository · arXiv:2106.13264Syntology ran 0 of 1 samples · 1 unverified
-
SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data 29 Mar 2021 · 2 repositories · arXiv:2103.15619Syntology ran 17 of 27 samples · 10 unverified
-
Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction 19 Feb 2021 · 2 repositories · arXiv:2104.00563Syntology ran 2 of 17 samples · 15 unverified
-
Gaining Insight into SARS-CoV-2 Infection and COVID-19 Severity Using Self-supervised Edge Features and Graph Neural Networks 23 Jun 2020 · 1 repository · arXiv:2006.12971
-
Self-supervised edge features for improved Graph Neural Network training 23 Jun 2020 · 1 repository · arXiv:2007.04777
-
Few-Shot Learning as Domain Adaptation: Algorithm and Analysis 6 Feb 2020 · 0 repositories · arXiv:2002.02050
-
Learning Set-equivariant Functions with SWARM Mappings 22 Jun 2019 · 1 repository · arXiv:1906.09400
-
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks 1 Oct 2018 · 9 repositories · arXiv:1810.00825Syntology ran 5 of 9 samples · 4 unverified · 4 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 48 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections