Methods › General › Stochastic Optimization › Lookahead
Lookahead
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Lookahead is a type of stochastic optimizer that iteratively updates two sets of weights: "fast" and "slow". Intuitively, the algorithm chooses a search direction by looking ahead at the sequence of fast weights generated by another optimizer.
Algorithm 1 Lookahead Optimizer
Require Initial parameters ϕ₀, objective function L
Require Synchronization period k, slow weights step size α, optimizer A
for t=1, 2, …
Synchronize parameters θ_(t,0) ϕₜ₋₁
for i=1, 2, …, k
sample minibatch of data d ∼𝒟
θ_(t,i) θ_(t,i-1) + A(L, θ_(t,i-1), d)
endfor
Perform outer update ϕₜ ϕₜ₋₁ + α(θ_(t,k) - ϕₜ₋₁)
endfor
return parameters ϕ
Papers archive 2025-07-28
21 shown of 21, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
AutoChemSchematic AI: A Closed-Loop, Physics-Aware Agentic Framework for Auto-Generating Chemical Process and Instrumentation Diagrams 30 May 2025 · 0 repositories · arXiv:2505.24584
-
Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query 24 May 2025 · 0 repositories · arXiv:2505.20334
-
Chronologically Consistent Large Language Models 28 Feb 2025 · 0 repositories · arXiv:2502.21206
-
Near Optimal Decision Trees in a SPLIT Second 21 Feb 2025 · 0 repositories · arXiv:2502.15988
-
Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree 17 Dec 2024 · 0 repositories · arXiv:2412.12639
-
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding 3 Feb 2024 · 1 repository · arXiv:2402.02057
-
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty 26 Jan 2024 · 1 repository · arXiv:2401.15077
-
AlphaMapleSAT: An MCTS-based Cube-and-Conquer SAT Solver for Hard Combinatorial Problems 24 Jan 2024 · 0 repositories · arXiv:2401.13770
-
Stable Nonconvex-Nonconcave Training via Linear Interpolation 20 Oct 2023 · 1 repository · arXiv:2310.13459Syntology ran 5 of 5 samples · 0 unverified · 5 pointer-only (licence)
-
Using fine-tuning and min lookahead beam search to improve Whisper 19 Sep 2023 · 0 repositories · arXiv:2309.10299
-
Exploring Flat Minima for Domain Generalization with Large Learning Rates 12 Sep 2023 · 0 repositories · arXiv:2309.06337
-
Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environments 5 Sep 2023 · 0 repositories · arXiv:2309.02328
-
On the Depth between Beam Search and Exhaustive Search for Text Generation 25 Aug 2023 · 0 repositories · arXiv:2308.13696
-
Lookbehind-SAM: k steps back, 1 step forward 31 Jul 2023 · 1 repository · arXiv:2307.16704Syntology ran 3 of 3 samples · 0 unverified
-
Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks 6 Mar 2023 · 0 repositories · arXiv:2303.03540
-
Multilayer Lookahead: a Nested Version of Lookahead 27 Oct 2021 · 0 repositories · arXiv:2110.14254
-
Characterizing Lookahead Dynamics of Smooth Games 1 Jan 2021 · 0 repositories
-
Taming GANs with Lookahead-Minmax 25 Jun 2020 · 1 repository · arXiv:2006.14567
-
sEMG Gesture Recognition with a Simple Model of Attention 5 Jun 2020 · 2 repositories · arXiv:2006.03645Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
-
Train-by-Reconnect: Decoupling Locations of Weights from their Values 5 Mar 2020 · 1 repository · arXiv:2003.02570Syntology ran 0 of 1 samples · 1 unverified
-
Lookahead Optimizer: k steps forward, 1 step back 19 Jul 2019 · 19 repositories · arXiv:1907.08610
Tasks archive 2025-07-28
20 shown of 27 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections