Papers › Generalization without systematicity: On the compositional skills of...

Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks

31 Oct 2017ICML 2018 7arXiv:1711.00350archive 2025-07-28

Brenden M. Lake, Marco Baroni

Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb "dax," he or she can immediately understand the meaning of "dax twice" or "sing and dax." In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply "mix-and-match" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the "dax" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

brendenlake/SCAN officialmentioned in papermentioned on GitHubNOASSERTION report

JanAthmer/Compositional-generalization-capabillity-of-Transformer mentioned on GitHubpytorch report

aman313/SCAN mentioned on GitHubpytorchNOASSERTION report

arkilpatel/compositional-generalization-seq2seq mentioned on GitHubpytorchMIT report

i-machine-think/machine-tasks mentioned on GitHub report

maxwells-daemons/compositional-learning-experiments mentioned on GitHubpytorchApache-2.0 report

yoonkim/neural-qcfg mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Machine Translation Translation Zero-shot Generalization

Datasets

Introduced by this paper, per the archive.

SCAN

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections