Methods › General › Stochastic Optimization › Lookahead
Lookahead
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Lookahead is a type of stochastic optimizer that iteratively updates two sets of weights: "fast" and "slow". Intuitively, the algorithm chooses a search direction by looking ahead at the sequence of fast weights generated by another optimizer.
Algorithm 1 Lookahead Optimizer
Require Initial parameters ϕ₀, objective function L
Require Synchronization period k, slow weights step size α, optimizer A
for t=1, 2, …
Synchronize parameters θ_(t,0) ←ϕₜ₋₁
for i=1, 2, …, k
sample minibatch of data d ∼𝒟
θ_(t,i) ←θ_(t,i-1) + A(L, θ_(t,i-1), d)
endfor
Perform outer update ϕₜ ←ϕₜ₋₁ + α(θ_(t,k) - ϕₜ₋₁)
endfor
return parameters ϕ
In Syntology repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). The lookup matched 'LOOKAHEAD' ignoring case. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2002.04809) is about this method is not checked.
From the lookup's note on this name: The same name also belongs to the paper “Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy” (arXiv:2312.12728, 51 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name. For arXiv:2312.12728: its page on this site · ask the lookup about it.
Origin paper of this answer, arXiv:2002.04809 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 6 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 21 · with a code link: 8 · where Syntology ran a sample: 3 (3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3 of 21 tagged: 3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 21 of 21, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
AutoChemSchematic AI: A Closed-Loop, Physics-Aware Agentic Framework for Auto-Generating Chemical Process and Instrumentation Diagrams 30 May 2025 · 0 repositories · arXiv:2505.24584
-
Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query 24 May 2025 · 0 repositories · arXiv:2505.20334
-
Chronologically Consistent Large Language Models 28 Feb 2025 · 0 repositories · arXiv:2502.21206
-
Near Optimal Decision Trees in a SPLIT Second 21 Feb 2025 · 0 repositories · arXiv:2502.15988
-
Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree 17 Dec 2024 · 0 repositories · arXiv:2412.12639
-
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding 3 Feb 2024 · 1 repository · arXiv:2402.02057
-
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty 26 Jan 2024 · 1 repository · arXiv:2401.15077
-
AlphaMapleSAT: An MCTS-based Cube-and-Conquer SAT Solver for Hard Combinatorial Problems 24 Jan 2024 · 0 repositories · arXiv:2401.13770
-
Stable Nonconvex-Nonconcave Training via Linear Interpolation 20 Oct 2023 · 1 repository · arXiv:2310.13459Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Using fine-tuning and min lookahead beam search to improve Whisper 19 Sep 2023 · 0 repositories · arXiv:2309.10299
-
Exploring Flat Minima for Domain Generalization with Large Learning Rates 12 Sep 2023 · 0 repositories · arXiv:2309.06337
-
Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environments 5 Sep 2023 · 0 repositories · arXiv:2309.02328
-
On the Depth between Beam Search and Exhaustive Search for Text Generation 25 Aug 2023 · 0 repositories · arXiv:2308.13696
-
Lookbehind-SAM: k steps back, 1 step forward 31 Jul 2023 · 1 repository · arXiv:2307.16704Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 3 harvested samples)
-
Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks 6 Mar 2023 · 0 repositories · arXiv:2303.03540
-
Multilayer Lookahead: a Nested Version of Lookahead 27 Oct 2021 · 0 repositories · arXiv:2110.14254
-
Characterizing Lookahead Dynamics of Smooth Games 1 Jan 2021 · 0 repositories
-
Taming GANs with Lookahead-Minmax 25 Jun 2020 · 1 repository · arXiv:2006.14567
-
sEMG Gesture Recognition with a Simple Model of Attention 5 Jun 2020 · 2 repositories · arXiv:2006.03645Syntology official: no sample here; runs from other or unrecorded repositories · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Train-by-Reconnect: Decoupling Locations of Weights from their Values 5 Mar 2020 · 1 repository · arXiv:2003.02570Syntology 0 ran · 1 unverified (of 1 harvested sample)
-
Lookahead Optimizer: k steps forward, 1 step back 19 Jul 2019 · 19 repositories · arXiv:1907.08610
Tasks archive 2025-07-28
20 shown of 27 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections