Methods › General › Attention Mechanisms › Set Transformer
Set Transformer
Introduced by Juho Lee et al. in Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set, models used to address them should be permutation invariant. We present an attention-based neural network module, the Set Transformer, specifically designed to model interactions among elements in the input set. The model consists of an encoder and a decoder, both of which rely on attention mechanisms. In an effort to reduce computational complexity, we introduce an attention scheme inspired by inducing point methods from sparse Gaussian process literature. It reduces the computation time of self-attention from quadratic to linear in the number of elements in the set. We show that our model is theoretically attractive and we evaluate it on a range of tasks, demonstrating the state-of-the-art performance compared to recent methods for set-structured data.
In Syntology the paper matched by title, the paper Syntology's graph keys as openreview_Hkgnii09Ym, held, no code (no_code; paper_only). The lookup matched a paper by its title, the paper Syntology's graph keys as openreview_Hkgnii09Ym, not a method entry. The archive credits no paper with an arXiv id for this method, so whether this answer (about the paper Syntology's graph keys as openreview_Hkgnii09Ym) is about this method is not checked; it does not say whether this method has code.
From the lookup's note on this name: The same name also belongs to the paper “Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks” (arXiv:1810.00825, 266 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name. For arXiv:1810.00825: its page on this site · ask the lookup about it.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 20 · with a code link: 14 · where Syntology ran a sample: 6 (6 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 20 tagged: 6 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 20 of 20, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
To Bin or not to Bin: Alternative Representations of Mass Spectra 15 Feb 2025 · 0 repositories · arXiv:2502.10851
-
Advances in Set Function Learning: A Survey of Techniques and Applications 24 Jan 2025 · 0 repositories · arXiv:2501.14991
-
Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks 20 Jan 2025 · 0 repositories · arXiv:2501.11350
-
Multiset Transformer: Advancing Representation Learning in Persistence Diagrams 22 Nov 2024 · 1 repository · arXiv:2411.14662
-
Graph as Point Set 5 May 2024 · 1 repository · arXiv:2405.02795Syntology official (archive's flag): 20 ran · 20 ran (of which 15 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 0 violated, 15 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified (of 27 harvested samples) · 27 pointer-only (licence)
-
Associative Transformer 22 Sep 2023 · 1 repository · arXiv:2309.12862
-
Pointersect: Neural Rendering with Cloud-Ray Intersection 24 Apr 2023 · 0 repositories · arXiv:2304.12390
-
Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams 7 Mar 2023 · 1 repository · arXiv:2303.03856
-
Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets 23 Jun 2022 · 1 repository · arXiv:2206.11925Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples)
-
Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation 11 Apr 2022 · 0 repositories · arXiv:2204.04913
-
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds 19 Mar 2022 · 1 repository · arXiv:2203.10314Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)
-
Automated Identification of Cell Populations in Flow Cytometry Data with Transformers 23 Aug 2021 · 1 repository · arXiv:2108.10072
-
You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks 24 Jun 2021 · 1 repository · arXiv:2106.13264Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data 29 Mar 2021 · 2 repositories · arXiv:2103.15619Syntology official (archive's flag): 17 ran · 17 ran (of which 7 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 8 where Syntology's instrument failed) · 10 unverified (of 27 harvested samples)
-
Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction 19 Feb 2021 · 2 repositories · arXiv:2104.00563Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified (of 17 harvested samples) · 2 pointer-only (licence)
-
Gaining Insight into SARS-CoV-2 Infection and COVID-19 Severity Using Self-supervised Edge Features and Graph Neural Networks 23 Jun 2020 · 1 repository · arXiv:2006.12971
-
Self-supervised edge features for improved Graph Neural Network training 23 Jun 2020 · 1 repository · arXiv:2007.04777
-
Few-Shot Learning as Domain Adaptation: Algorithm and Analysis 6 Feb 2020 · 0 repositories · arXiv:2002.02050
-
Learning Set-equivariant Functions with SWARM Mappings 22 Jun 2019 · 1 repository · arXiv:1906.09400
-
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks 1 Oct 2018 · 9 repositories · arXiv:1810.00825Syntology official (archive's flag): 3 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 3 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 4 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 48 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections