{"url":"/method/sarsa","slug":"sarsa","name":"Sarsa","full_name":"Sarsa","full_name_withheld":false,"description_markdown":"**Sarsa** is an on-policy TD control algorithm:\r\n\r\n$$Q\\left(S\\_{t}, A\\_{t}\\right) \\leftarrow Q\\left(S\\_{t}, A\\_{t}\\right) + \\alpha\\left[R_{t+1} + \\gamma{Q}\\left(S\\_{t+1}, A\\_{t+1}\\right) - Q\\left(S\\_{t}, A\\_{t}\\right)\\right] $$\r\n\r\nThis update is done after every transition from a nonterminal state $S\\_{t}$. if $S\\_{t+1}$ is terminal, then $Q\\left(S\\_{t+1}, A\\_{t+1}\\right)$ is defined as zero.\r\n\r\nTo design an on-policy control algorithm using Sarsa, we estimate $q\\_{\\pi}$ for a behaviour policy $\\pi$ and then change $\\pi$ towards greediness with respect to $q\\_{\\pi}$.\r\n\r\nSource: Sutton and Barto, Reinforcement Learning, 2nd Edition","description_state":"present","introduced_year":1994,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"On-Policy TD Control","url":"/methods/category/on-policy-td-control","pwc_aliases":[]}],"n_papers_tagged":56,"archive_num_papers":56,"papers_newest_first":[{"paper":null,"title":"A Unified Anti-Jamming Design in Complex Environments Based on Cross-Modal Fusion and Intelligent Decision-Making","date":"2025-06-09","arxiv_id":"2506.07532","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning for Hanabi","date":"2025-05-31","arxiv_id":"2506.00458","n_code_links":0,"syntology":null},{"paper":null,"title":"Convergent NMPC-based Reinforcement Learning Using Deep Expected Sarsa and Nonlinear Temporal Difference Learning","date":"2025-02-07","arxiv_id":"2502.04925","n_code_links":0,"syntology":null},{"paper":null,"title":"Segmenting Action-Value Functions Over Time-Scales in SARSA via TD($Δ$)","date":"2024-11-22","arxiv_id":"2411.14783","n_code_links":0,"syntology":null},{"paper":null,"title":"A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robot","date":"2024-09-23","arxiv_id":"2409.14867","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning for Rate Maximization in IRS-aided OWC Networks","date":"2024-09-07","arxiv_id":"2409.04842","n_code_links":0,"syntology":null},{"paper":null,"title":"Optimally Solving Simultaneous-Move Dec-POMDPs: The Sequential Central Planning Approach","date":"2024-08-23","arxiv_id":"2408.13139","n_code_links":0,"syntology":null},{"paper":null,"title":"The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game","date":"2024-06-25","arxiv_id":"2406.17326","n_code_links":0,"syntology":null},{"paper":"/paper/swiftrl-towards-efficient-reinforcement","title":"SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems","date":"2024-05-07","arxiv_id":"2405.03967","n_code_links":1,"syntology":null},{"paper":"/paper/research-on-robot-path-planning-based-on","title":"Research on Robot Path Planning Based on Reinforcement Learning","date":"2024-04-22","arxiv_id":"2404.14077","n_code_links":1,"syntology":null},{"paper":null,"title":"State-Separated SARSA: A Practical Sequential Decision-Making Algorithm with Recovering Rewards","date":"2024-03-18","arxiv_id":"2403.11520","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing Classification Performance via Reinforcement Learning for Feature Selection","date":"2024-03-09","arxiv_id":"2403.05979","n_code_links":0,"syntology":null},{"paper":null,"title":"An Index Policy Based on Sarsa and Q-learning for Heterogeneous Smart Target Tracking","date":"2024-02-19","arxiv_id":"2402.12015","n_code_links":0,"syntology":null},{"paper":null,"title":"Using Reinforcement Learning to Optimize Responses in Care Processes: A Case Study on Aggression Incidents","date":"2023-10-02","arxiv_id":"2310.00981","n_code_links":0,"syntology":null},{"paper":null,"title":"Career Path Recommendations for Long-term Income Maximization: A Reinforcement Learning Approach","date":"2023-09-11","arxiv_id":"2309.05391","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-reinforcement-learning-techniques","title":"Exploring reinforcement learning techniques for discrete and continuous control tasks in the MuJoCo environment","date":"2023-07-20","arxiv_id":"2307.11166","n_code_links":1,"syntology":null},{"paper":null,"title":"PCG-based Static Underground Garage Scenario Generation","date":"2023-07-08","arxiv_id":"2307.03988","n_code_links":0,"syntology":null},{"paper":null,"title":"Convergence of SARSA with linear function approximation: The random horizon case","date":"2023-06-07","arxiv_id":"2306.04548","n_code_links":0,"syntology":null},{"paper":null,"title":"On Modeling Network Slicing Communication Resources with SARSA Optimization","date":"2023-01-11","arxiv_id":"2301.04696","n_code_links":0,"syntology":null},{"paper":null,"title":"Analysis of Reinforcement Learning Schemes for Trajectory Optimization of an Aerial Radio Unit","date":"2022-11-18","arxiv_id":"2211.10524","n_code_links":0,"syntology":null},{"paper":null,"title":"SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions","date":"2022-09-05","arxiv_id":"2209.01876","n_code_links":0,"syntology":null},{"paper":"/paper/solving-royal-game-of-ur-using-reinforcement","title":"Solving Royal Game of Ur Using Reinforcement Learning","date":"2022-08-23","arxiv_id":"2208.10669","n_code_links":2,"syntology":null},{"paper":null,"title":"On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios","date":"2022-07-19","arxiv_id":"2207.09372","n_code_links":0,"syntology":null},{"paper":"/paper/reinforced-lin-kernighan-helsgaun-algorithms","title":"Reinforced Lin-Kernighan-Helsgaun Algorithms for the Traveling Salesman Problems","date":"2022-07-08","arxiv_id":"2207.03876","n_code_links":1,"syntology":null},{"paper":null,"title":"Does DQN Learn?","date":"2022-05-26","arxiv_id":"2205.13617","n_code_links":0,"syntology":null},{"paper":null,"title":"On the Convergence of SARSA with Linear Function Approximation","date":"2022-02-14","arxiv_id":"2202.06828","n_code_links":0,"syntology":null},{"paper":null,"title":"A study of first-passage time minimization via Q-learning in heated gridworlds","date":"2021-10-05","arxiv_id":"2110.02129","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing Video Analytics Accuracy via Real-time Automated Camera Parameter Tuning","date":"2021-07-08","arxiv_id":"2107.03964","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-for-phy-layer","title":"Reinforcement Learning for Physical Layer Communications","date":"2021-06-22","arxiv_id":"2106.11595","n_code_links":1,"syntology":null},{"paper":null,"title":"A Provably-Efficient Model-Free Algorithm for Constrained Markov Decision Processes","date":"2021-06-03","arxiv_id":"2106.01577","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":39},{"task":"/task/q-learning","name":"Q-Learning","papers":34},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":32},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":29},{"task":"/task/decision-making","name":"Decision Making","papers":5},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":5},{"task":"/task/continuous-control","name":"Continuous Control","papers":3},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":3},{"task":"/task/continuous-control","name":"continuous-control","papers":3},{"task":"/task/combinatorial-optimization","name":"Combinatorial Optimization","papers":2},{"task":"/task/management","name":"Management","papers":2},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":2},{"task":"/task/traveling-salesman-problem","name":"Traveling Salesman Problem","papers":2},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":1},{"task":"/task/board-games","name":"Board Games","papers":1},{"task":null,"name":"CPU","papers":1},{"task":"/task/card-games","name":"Card Games","papers":1},{"task":"/task/classification-1","name":"Classification","papers":1},{"task":"/task/dialogue-management","name":"Dialogue Management","papers":1},{"task":"/task/efficient-exploration","name":"Efficient Exploration","papers":1}],"tasks_shown":20,"n_tasks":38,"usage_by_year":[{"year":"2009","papers":1},{"year":"2015","papers":1},{"year":"2017","papers":4},{"year":"2018","papers":6},{"year":"2019","papers":5},{"year":"2020","papers":8},{"year":"2021","papers":5},{"year":"2022","papers":7},{"year":"2023","papers":6},{"year":"2024","papers":10},{"year":"2025","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sarsa"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}