{"url":"/method/replacing-eligibility-trace","slug":"replacing-eligibility-trace","name":"Replacing Eligibility Trace","full_name":"Replacing Eligibility Trace","full_name_withheld":false,"description_markdown":"In a **Replacing Eligibility Trace**, each time the state is revisited, the trace is reset to $1$ regardless of the presence of a prior trace.. For the memory vector $\\textbf{e}\\_{t} \\in \\mathbb{R}^{b} \\geq \\textbf{0}$:\r\n\r\n$$\\mathbf{e\\_{0}} = \\textbf{0}$$\r\n\r\n$$\\textbf{e}\\_{t} = \\gamma\\lambda{e}\\_{t-1}\\left(s\\right) \\text{ if } s \\neq s\\_{t}$$\r\n\r\n$$\\textbf{e}\\_{t} = 1 \\text{ if } s = s\\_{t}$$\r\n\r\nThey can be seen as crude approximation to dutch traces, which have largely superseded them as they perform better than replacing traces and have a clearer theoretical basis. Accumulating traces remain of interest for nonlinear function approximations where dutch traces are not available.\r\n\r\nSource: Sutton and Barto, Reinforcement Learning, 2nd Edition","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Eligibility Traces","url":"/methods/category/eligibility-traces","pwc_aliases":[]}],"n_papers_tagged":0,"archive_num_papers":0,"papers_newest_first":[],"papers_shown":0,"tasks":[],"tasks_shown":0,"n_tasks":0,"usage_by_year":[],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/replacing-eligibility-trace"},"syntology_read_at":"2026-09-25T09:33:49+00:00"}