Systematic Generalization

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

7 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 77 papers with code (126 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Multi-Object Representation Learning with Iterative Variational Inference
  1 Mar 2019 6 repositories listed Syntology ran 1 of 6 samples · 5 unverified
  Human perception is structured around objects which form the basis for our higher-level cognition and impressive systematic generalization abilities.
• CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
  16 Aug 2019 5 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way.
• Prioritized Level Replay
  8 Oct 2020 4 repositories listed
  Environments with procedurally generated content serve as important benchmarks for testing systematic generalization in deep reinforcement learning.
• A Benchmark for Systematic Generalization in Grounded Language Understanding
  11 Mar 2020 4 repositories listed
  In this paper, we introduce a new benchmark, gSCAN, for evaluating compositional generalization in situated language understanding.
• Conditional Object-Centric Learning from Video
  24 Nov 2021 3 repositories listed Syntology ran 11 of 13 samples · 2 unverified · 10 pointer-only (licence)
  Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built.
• The NetHack Learning Environment
  24 Jun 2020 3 repositories listed Syntology ran 1 of 11 samples · 10 unverified · 3 pointer-only (licence)
  Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike…
• CLOSURE: Assessing Systematic Generalization of CLEVR Models
  12 Dec 2019 3 repositories listed Syntology ran 5 of 10 samples · 5 unverified · 2 pointer-only (licence)
  In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.
• Slot Abstractors: Toward Scalable Abstract Visual Reasoning
  6 Mar 2024 2 repositories listed Syntology ran 2 of 4 samples · 2 unverified
  Abstract visual reasoning is a characteristically human ability, allowing the identification of relational patterns that are abstracted away from object features, and the systematic generalization of those patterns to…
• Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts
  31 May 2023 2 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  Neuro-Symbolic (NeSy) predictive models hold the promise of improved compliance with given constraints, systematic generalization, and interpretability, as they allow to infer labels that are consistent with some prior…
• VIMA: General Robot Manipulation with Multimodal Prompts
  6 Oct 2022 2 repositories listed Syntology ran 0 of 8 samples · 8 unverified
  We show that a wide spectrum of robot manipulation tasks can be expressed with multimodal prompts, interleaving textual and visual tokens.
• Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
  25 Sep 2021 2 repositories listed Syntology ran 1 of 7 samples · 6 unverified
  We analyze the grounded SCAN (gSCAN) benchmark, which was recently proposed to study systematic generalization for grounded language understanding.
• The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers
  26 Aug 2021 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  Our models improve accuracy from 50% to 85% on the PCFG productivity split, and from 35% to 81% on COGS.
• Measuring Systematic Generalization in Neural Proof Generation with Transformers
  30 Sep 2020 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  We observe that models that are not trained to generate proofs are better at generalizing to problems based on longer proofs.
• Systematic Generalization on gSCAN with Language Conditioned Embedding
  11 Sep 2020 2 repositories listed
  Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations that are distinct but semantically similar to its training data.
• Revisit Systematic Generalization via Meaningful Learning
  14 Mar 2020 2 repositories listed
  Humans can systematically generalize to novel compositions of existing concepts.
• Systematic Generalization: What Is Required and Can It Be Learned?
  30 Nov 2018 2 repositories listed Syntology ran 5 of 5 samples · 0 unverified · 5 pointer-only (licence)
  Numerous models for grounded language understanding have been recently proposed, including (i) generic models that can be easily adapted to any given task and (ii) intuitively appealing modular models that require…
• Enabling Systematic Generalization in Abstract Spatial Reasoning through Meta-Learning for Compositionality
  2 Apr 2025 1 repository listed
  To this end, we introduce SYGAR-a dataset designed to evaluate the capacity of models to systematically generalize from known geometric transformations (e.
• Unveiling the Mechanisms of Explicit CoT Training: How CoT Enhances Reasoning Generalization
  7 Feb 2025 1 repository listed Syntology ran 2 of 10 samples · 8 unverified
  \textbf{1)} Structural Advantage: CoT training internalizes reasoning into a two-stage generalizing circuit, where the number of stages corresponds to the explicit reasoning steps during training.
• Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning
  25 Jan 2025 1 repository listed
  However, to transfer these properties of language to the decision-making process, it is necessary to establish a proper language grounding mechanism.
• CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives
  29 Nov 2024 1 repository listed
  Grounding the instruction in the environment is a key step in solving language-guided goal-reaching reinforcement learning problems.
• Many-body Expansion Based Machine Learning Models for Octahedral Transition Metal Complexes
  12 Oct 2024 1 repository listed
  On a test set composed of all possible isomers of binary transition metal complexes, the best MBE models achieve mean absolute errors of 2.
• On The Specialization of Neural Modules
  23 Sep 2024 1 repository listed Syntology ran 4 of 11 samples · 7 unverified
  Our results shed light on the difficulty of module specialization, what is required for modules to successfully specialize, and the necessity of modular architectures to achieve systematicity.
• Systematic Reasoning About Relational Domains With Graph Neural Networks
  24 Jul 2024 1 repository listed Syntology ran 9 of 10 samples · 1 unverified
  A common solution is to instead rely on neuro-symbolic methods, which are capable of reasoning in a systematic way by design.
• A Systematization of the Wagner Framework: Graph Theory Conjectures and Reinforcement Learning
  18 Jun 2024 1 repository listed
  The game ends when all edges have been considered, resulting in a certain graph G_T, and f(G_T) is the final score of the game; RL is then used to maximize this score.
• Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention
  11 Jun 2024 1 repository listed Syntology ran 0 of 2 samples · 2 unverified
  Learning modular object-centric representations is crucial for systematic generalization.
• Discrete Dictionary-based Decomposition Layer for Structured Representation Learning
  11 Jun 2024 1 repository listed Syntology ran 10 of 14 samples · 4 unverified
  To address this decomposition problem, we propose a Discrete Dictionary-based Decomposition (D3) layer designed to enhance the decomposition capabilities of TPR-based models.
• Attention-based Iterative Decomposition for Tensor Product Representation
  3 Jun 2024 1 repository listed Syntology ran 4 of 4 samples · 0 unverified
  In recent research, Tensor Product Representation (TPR) is applied for the systematic generalization task of deep neural networks by learning the compositional structure of data.
• Mixture of Latent Experts Using Tensor Products
  26 May 2024 1 repository listed
  For \textit{routing function}, we tailor two innovative routing functions according to the granularity: \texttt{TensorPoly-I} which directs to each rank within the entangled tensor while \texttt{TensorPoly-II} offers a…
• Learning to Compose: Improving Object Centric Learning by Injecting Compositionality
  1 May 2024 1 repository listed
  Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning.
• Case-Based or Rule-Based: How Do Transformers Do the Math?
  27 Feb 2024 1 repository listed Syntology ran 13 of 13 samples · 0 unverified
  Through carefully designed intervention experiments on five math tasks, we confirm that transformers are performing case-based reasoning, no matter whether scratchpad is used, which aligns with the previous observations…

Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.