Methods › Natural Language Processing › Transformers › ETC

Extended Transformer Construction

ETC

36 papers tagged archive 2025-07-28

Introduced by Joshua Ainslie et al. in ETC: Encoding Long and Structured Inputs in Transformers

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Extended Transformer Construction, or ETC, is an extension of the Transformer architecture with a new attention mechanism that extends the original in two main ways: (1) it allows scaling up the input length from 512 to several thousands; and (2) it can ingesting structured inputs instead of just linear sequences. The key ideas that enable ETC to achieve these are a new global-local attention mechanism, coupled with relative position encodings. ETC also allows lifting weights from existing BERT models, saving computational resources while training.

Paper Source

Papers archive 2025-07-28

30 shown of 36, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Two-Player Zero-Sum Games with Bandit Feedback
17 Jun 2025 · 0 repositories · arXiv:2506.14518
Koopman-Based Event-Triggered Control from Data
19 Apr 2025 · 0 repositories · arXiv:2504.14334
Performance-Barrier Event-Triggered PDE Control of Traffic Flow
1 Jan 2025 · 0 repositories · arXiv:2501.00722
Automated Toll Management System Using RFID and Image Processing
2 Dec 2024 · 0 repositories · arXiv:2412.01728
Hierarchical Event-Triggered Systems: Safe Learning of Quasi-Optimal Deadline Policies
15 Sep 2024 · 0 repositories · arXiv:2409.09812
Performance-Barrier Event-Triggered Control of a Class of Reaction-Diffusion PDEs
11 Jul 2024 · 0 repositories · arXiv:2407.08178
Contextual Dynamic Pricing: Algorithms, Optimality, and Local Differential Privacy Constraints
4 Jun 2024 · 0 repositories · arXiv:2406.02424
Event-Triggered Robust Cooperative Output Regulation for a Class of Linear Multi-Agent Systems with an Unknown Exosystem
1 Mar 2024 · 0 repositories · arXiv:2403.00645
Replication-proof Bandit Mechanism Design with Bayesian Agents
28 Dec 2023 · 0 repositories · arXiv:2312.16896
Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks
24 Oct 2023 · 0 repositories · arXiv:2310.15705
Listen to Minority: Encrypted Traffic Classification for Class Imbalance with Contrastive Pre-Training
31 Aug 2023 · 0 repositories · arXiv:2308.16453
High-dimensional Contextual Bandit Problem without Sparsity
19 Jun 2023 · 0 repositories · arXiv:2306.11017
Permutation Decision Trees
5 Jun 2023 · 0 repositories · arXiv:2306.02617
An Improved Heart Disease Prediction Using Stacked Ensemble Method
12 Apr 2023 · 0 repositories · arXiv:2304.06015
Asynchronous Event-Triggered Control for Non-Linear Systems
25 Nov 2022 · 0 repositories · arXiv:2211.13846
LittleBird: Efficient Faster & Longer Transformer for Question Answering
21 Oct 2022 · 0 repositories · arXiv:2210.11870
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding
8 Oct 2022 · 0 repositories · arXiv:2210.03915
Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency
26 May 2022 · 0 repositories · arXiv:2205.13476
Thompson Sampling for Bandit Learning in Matching Markets
26 Apr 2022 · 1 repository · arXiv:2204.12048
ETCetera: beyond Event-Triggered Control
3 Mar 2022 · 0 repositories · arXiv:2203.01623
Formal Analysis of the Sampling Behaviour of Stochastic Event-Triggered Control
21 Feb 2022 · 0 repositories · arXiv:2202.10178
Chaos and order in event-triggered control
12 Jan 2022 · 0 repositories · arXiv:2201.04462
Detecting Extratropical Cyclones of the Northern Hemisphere with Single Shot Detector
1 Dec 2021 · 0 repositories · arXiv:2112.01283
Bandits with Dynamic Arm-acquisition Costs
23 Oct 2021 · 0 repositories · arXiv:2110.12118
Computing the average inter-sample time of event-triggered control using quantitative automata
29 Sep 2021 · 0 repositories · arXiv:2109.14391
Uncertainties and output feedback in rollout event-triggered control
20 Aug 2021 · 0 repositories · arXiv:2108.09125
Rollout event-triggered control: reconciling event- and time-triggered control
6 Aug 2021 · 0 repositories · arXiv:2108.02994
Debiasing Samples from Online Learning Using Bootstrap
31 Jul 2021 · 0 repositories · arXiv:2108.00236
Model Selection for Generic Contextual Bandits
7 Jul 2021 · 0 repositories · arXiv:2107.03455
Abstracting the Sampling Behaviour of Stochastic Linear Periodic Event-Triggered Control Systems
25 Mar 2021 · 0 repositories · arXiv:2103.13839

Tasks archive 2025-07-28

20 shown of 38 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

Task	Papers
Scheduling	3
Language Modelling	2
Multi-Armed Bandits	2
Question Answering	2
Reinforcement Learning (RL)	2
Thompson Sampling	2
reinforcement-learning	2
BIG-bench Machine Learning	1
Deep Reinforcement Learning	1
Diagnostic	1
Disease Prediction	1
Language Modeling	1
Linguistic Acceptability	1
Machine Translation	1
Management	1
Masked Language Modeling	1
Model Predictive Control	1
Model Selection	1
Natural Language Inference	1
Natural Questions	1

Usage over time archive 2025-07-28

Papers per year the archive tags with this method, by the paper's archive date (36 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Transformers

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections