{"url":"/method/sarsa-lambda","slug":"sarsa-lambda","name":"Sarsa Lambda","full_name":"Sarsa Lambda","full_name_withheld":false,"description_markdown":"**Sarsa_INLINE_MATH_1** extends eligibility-traces to action-value methods. It has the same update rule as for **TD_INLINE_MATH_1** but we use the action-value form of the TD erorr:\r\n\r\n$$ \\delta\\_{t} = R\\_{t+1} + \\gamma\\hat{q}\\left(S\\_{t+1}, A\\_{t+1}, \\mathbb{w}\\_{t}\\right) - \\hat{q}\\left(S\\_{t}, A\\_{t}, \\mathbb{w}\\_{t}\\right) $$\r\n\r\nand the action-value form of the [eligibility trace](https://paperswithcode.com/method/eligibility-trace):\r\n\r\n$$ \\mathbb{z}\\_{-1} = \\mathbb{0} $$\r\n\r\n$$ \\mathbb{z}\\_{t} = \\gamma\\lambda\\mathbb{z}\\_{t-1} + \\nabla\\hat{q}\\left(S\\_{t}, A\\_{t}, \\mathbb{w}\\_{t} \\right), 0 \\leq t \\leq T$$\r\n\r\nSource: Sutton and Barto, Reinforcement Learning, 2nd Edition","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"On-Policy TD Control","url":"/methods/category/on-policy-td-control","pwc_aliases":[]}],"n_papers_tagged":0,"archive_num_papers":0,"papers_newest_first":[],"papers_shown":0,"tasks":[],"tasks_shown":0,"n_tasks":0,"usage_by_year":[],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sarsa-lambda"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}