Methods › Reinforcement Learning › On-Policy TD Control › TD Lambda
TD Lambda
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
TD_INLINE_MATH_1 is a generalisation of TD_INLINE_MATH_2 reinforcement learning algorithms, but it employs an eligibility trace λ and λ-weighted returns. The eligibility trace vector is initialized to zero at the beginning of the episode, and it is incremented on each time step by the value gradient, and then fades away by γλ:
z₋₁ = 0 zₜ = γλzₜ₋₁ + ∇v̂(Sₜ, 𝐰ₜ), 0 ≤t ≤T
The eligibility trace keeps track of which components of the weight vector contribute to recent state valuations. Here ∇v̂(Sₜ, 𝐰ₜ) is the feature vector.
The TD error for state-value prediction is:
δₜ = Rₜ₊₁ + γv̂Sₜ₊₁, 𝐰ₜ) - v̂(Sₜ, 𝐰ₜ)
In TD_INLINE_MATH_1, the weight vector is updated on each step proportional to the scalar TD error and the vector eligibility trace:
𝐰ₜ₊₁ = 𝐰ₜ + αδ𝐳ₜ
Source: Sutton and Barto, Reinforcement Learning, 2nd Edition
Papers archive 2025-07-28
14 shown of 14, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
On-line Policy Improvement using Monte-Carlo Search 9 Jan 2025 · 0 repositories · arXiv:2501.05407
-
Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming 2 Jun 2024 · 0 repositories · arXiv:2406.00592
-
Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach 19 Dec 2023 · 1 repository · arXiv:2312.11865Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
-
A Robust and Opponent-Aware League Training Method for StarCraft II 21 Sep 2023 · 0 repositories
-
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning 7 Aug 2023 · 1 repository · arXiv:2308.03526Syntology ran 4 of 4 samples · 0 unverified
-
On Efficient Reinforcement Learning for Full-length Game of StarCraft II 23 Sep 2022 · 2 repositories · arXiv:2209.11553Syntology ran 0 of 10 samples · 10 unverified
-
AI in Human-computer Gaming: Techniques, Challenges and Opportunities 15 Nov 2021 · 0 repositories · arXiv:2111.07631
-
Search in Imperfect Information Games 10 Nov 2021 · 0 repositories · arXiv:2111.05884
-
Rethinking of AlphaStar 7 Aug 2021 · 2 repositories · arXiv:2108.03452
-
An Introduction of mini-AlphaStar 14 Apr 2021 · 1 repository · arXiv:2104.06890
-
Deep Reinforcement Learning with Function Properties in Mean Reversion Strategies 9 Jan 2021 · 1 repository · arXiv:2101.03418
-
TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game 27 Nov 2020 · 1 repository · arXiv:2011.13729
-
AlphaStar: An Evolutionary Computation Perspective 5 Feb 2019 · 0 repositories · arXiv:1902.01724
-
A Hierarchical Reinforcement Learning Method for Persistent Time-Sensitive Tasks 20 Jun 2016 · 0 repositories · arXiv:1606.06355
Tasks archive 2025-07-28
17 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections