| GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling |
3 |
1 |
3 Nov 2023 |
official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified |
| Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation |
1 |
1 |
31 May 2023 |
official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (1 pointer-only for licence) |
| Hyena Hierarchy: Towards Larger Convolutional Language Models |
7 |
2 |
21 Feb 2023 |
official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (4 pointer-only for licence) |
| Hungry Hungry Hippos: Towards Language Modeling with State Space Models |
3 |
5 |
28 Dec 2022 |
official (archive's flag): 1 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified |
| You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LM |
1 |
1 |
28 Oct 2022 |
official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (3 pointer-only for licence) |
| Mega: Moving Average Equipped Gated Attention |
7 |
1 |
21 Sep 2022 |
official (archive's flag): 3 ran · 12 ran (of which 6 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (11 pointer-only for licence) |
| General-purpose, long-context autoregressive modeling with Perceiver AR |
3 |
1 |
15 Feb 2022 |
official (archive's flag): 1 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (1 pointer-only for licence) |
| Improving language models by retrieving from trillions of tokens |
2 |
1 |
8 Dec 2021 |
16 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 3 violated, 11 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (3 pointer-only for licence) |
| Efficiently Modeling Long Sequences with Structured State Spaces |
8 |
1 |
31 Oct 2021 |
official (archive's flag): 5 ran · 37 ran (of which 8 constructed an object rather than computing a result; 21 with no instrument failure: 2 honoured, 1 violated, 18 with no contract checked; 16 where Syntology's instrument failed) · 18 unverified (3 pointer-only for licence) |
| Revisiting Simple Neural Probabilistic Language Models |
1 |
1 |
8 Apr 2021 |
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified |
| Finetuning Pretrained Transformers into RNNs |
2 |
1 |
24 Mar 2021 |
3 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (4 pointer-only for licence) |
| GLM: General Language Model Pretraining with Autoregressive Blank Infilling |
8 |
2 |
18 Mar 2021 |
community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified |
| Rethinking Attention with Performers |
7 |
1 |
30 Sep 2020 |
official: harvested, nothing ran · 11 ran (of which 3 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (6 pointer-only for licence) |
| DeLighT: Deep and Light-weight Transformer |
2 |
1 |
3 Aug 2020 |
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified |
| Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention |
8 |
1 |
29 Jun 2020 |
official: no sample here; runs from other or unrecorded repositories · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (3 pointer-only for licence) |
| Efficient Content-Based Sparse Attention with Routing Transformers |
2 |
1 |
12 Mar 2020 |
3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 2 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence) |
| Addressing Some Limitations of Transformers with Feedback Memory |
4 |
2 |
21 Feb 2020 |
official: no sample here; runs from other or unrecorded repositories · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence) |
| Time-aware Large Kernel Convolutions |
1 |
1 |
8 Feb 2020 |
official (archive's flag): 6 ran · 6 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (3 pointer-only for licence) |
| Reformer: The Efficient Transformer |
10 |
1 |
13 Jan 2020 |
community repositories only · 6 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified |
| Compressive Transformers for Long-Range Sequence Modelling |
6 |
1 |
13 Nov 2019 |
6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified |
| Generalization through Memorization: Nearest Neighbor Language Models |
5 |
2 |
1 Nov 2019 |
community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified |
| On the adequacy of untuned warmup for adaptive optimization |
1 |
1 |
9 Oct 2019 |
4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified |
| Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism |
10 |
1 |
17 Sep 2019 |
community repositories only · 12 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 35 unverified (15 pointer-only for licence) |
| Deep Equilibrium Models |
11 |
3 |
3 Sep 2019 |
community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (4 pointer-only for licence) |
| Augmenting Self-attention with Persistent Memory |
2 |
1 |
2 Jul 2019 |
5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 3 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (4 pointer-only for licence) |
| Language Models with Transformers |
1 |
1 |
20 Apr 2019 |
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence) |
| Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context |
37 |
2 |
9 Jan 2019 |
official (archive's flag): 8 ran · 67 ran (of which 37 constructed an object rather than computing a result; 49 with no instrument failure: 4 honoured, 1 violated, 44 with no contract checked; 18 where Syntology's instrument failed) · 76 unverified (43 pointer-only for licence) |
| Trellis Networks for Sequence Modeling |
1 |
1 |
15 Oct 2018 |
official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence) |
| Relational recurrent neural networks |
2 |
1 |
5 Jun 2018 |
5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence) |
| An Analysis of Neural Language Modeling at Multiple Scales |
12 |
1 |
22 Mar 2018 |
community repositories only · 14 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 2 honoured, 0 violated, 11 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (4 pointer-only for licence) |
| An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling |
35 |
1 |
4 Mar 2018 |
community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence) |