Methods › Reinforcement Learning › Eligibility Traces › Eligibility Trace
Eligibility Trace
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
An Eligibility Trace is a memory vector zₜ ∈ℝᵈ that parallels the long-term weight vector wₜ ∈ℝᵈ. The idea is that when a component of wₜ participates in producing an estimated value, the corresponding component of zₜ is bumped up and then begins to fade away. Learning will then occur in that component of wₜ if a nonzero TD error occurs before the trade falls back to zero. The trace-decay parameter λ∈[0, 1] determines the rate at which the trace falls.
Intuitively, they tackle the credit assignment problem by capturing both a frequency heuristic - states that are visited more often deserve more credit - and a recency heuristic - states that are visited more recently deserve more credit.
E₀(s) = 0 Eₜ(s) = γλEₜ₋₁(s) + 1(Sₜ = s)
Source: Sutton and Barto, Reinforcement Learning, 2nd Edition
Papers archive 2025-07-28
11 shown of 11, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Noise-based reward-modulated learning 31 Mar 2025 · 0 repositories · arXiv:2503.23972
-
Predecessor Features 1 Jun 2022 · 0 repositories · arXiv:2206.00303
-
META-Learning Eligibility Traces for More Sample Efficient Temporal Difference Learning 16 Jun 2020 · 1 repository · arXiv:2006.08906
-
Efficient Use of heuristics for accelerating XCS-based Policy Learning in Markov Games 26 May 2020 · 0 repositories · arXiv:2005.12553
-
Gradient Q$(σ, λ)$: A Unified Algorithm with Function Approximation for Reinforcement Learning 6 Sep 2019 · 0 repositories · arXiv:1909.02877
-
Gap-Increasing Policy Evaluation for Efficient and Noise-Tolerant Reinforcement Learning 18 Jun 2019 · 0 repositories · arXiv:1906.07586
-
META-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation 25 Apr 2019 · 2 repositories · arXiv:1904.11439Syntology ran 0 of 14 samples · 14 unverified
-
Metatrace Actor-Critic: Online Step-size Tuning by Meta-gradient Descent for Reinforcement Learning Control 10 May 2018 · 0 repositories · arXiv:1805.04514
-
A Unified Approach for Multi-step Temporal-Difference Learning with Eligibility Traces in Reinforcement Learning 9 Feb 2018 · 0 repositories · arXiv:1802.03171
-
A forward model at Purkinje cell synapses facilitates cerebellar anticipatory control 1 Dec 2016 · 0 repositories
-
Q(λ) with Off-Policy Corrections 16 Feb 2016 · 0 repositories · arXiv:1602.04951
Tasks archive 2025-07-28
7 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Reinforcement Learning | 6 |
| Reinforcement Learning (RL) | 6 |
| Meta-Learning | 3 |
| reinforcement-learning | 2 |
| Atari Games | 1 |
| Q-Learning | 1 |
| counterfactual | 1 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections