Methods › Reinforcement Learning › On-Policy TD Control › Sarsa
Sarsa
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Sarsa is an on-policy TD control algorithm:
Q(Sₜ, Aₜ) ←Q(Sₜ, Aₜ) + α[Rₜ₊₁ + γQ(Sₜ₊₁, Aₜ₊₁) - Q(Sₜ, Aₜ)]
This update is done after every transition from a nonterminal state Sₜ. if Sₜ₊₁ is terminal, then Q(Sₜ₊₁, Aₜ₊₁) is defined as zero.
To design an on-policy control algorithm using Sarsa, we estimate q_π for a behaviour policy π and then change π towards greediness with respect to q_π.
Source: Sutton and Barto, Reinforcement Learning, 2nd Edition
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: MI-SARSA; Implicit Q-Learning and SARSA.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 56 · with a code link: 11 · where Syntology ran a sample: 0 (0 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran: none of the 56 tagged papers has a Syntology run on record
Page 1 of 1: papers 1 to 56 of 56, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
A Unified Anti-Jamming Design in Complex Environments Based on Cross-Modal Fusion and Intelligent Decision-Making 9 Jun 2025 · 0 repositories · arXiv:2506.07532
-
Reinforcement Learning for Hanabi 31 May 2025 · 0 repositories · arXiv:2506.00458
-
Convergent NMPC-based Reinforcement Learning Using Deep Expected Sarsa and Nonlinear Temporal Difference Learning 7 Feb 2025 · 0 repositories · arXiv:2502.04925
-
Segmenting Action-Value Functions Over Time-Scales in SARSA via TD(Δ) 22 Nov 2024 · 0 repositories · arXiv:2411.14783
-
A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robot 23 Sep 2024 · 0 repositories · arXiv:2409.14867
-
Reinforcement Learning for Rate Maximization in IRS-aided OWC Networks 7 Sep 2024 · 0 repositories · arXiv:2409.04842
-
Optimally Solving Simultaneous-Move Dec-POMDPs: The Sequential Central Planning Approach 23 Aug 2024 · 0 repositories · arXiv:2408.13139
-
The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game 25 Jun 2024 · 0 repositories · arXiv:2406.17326
-
SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems 7 May 2024 · 1 repository · arXiv:2405.03967
-
Research on Robot Path Planning Based on Reinforcement Learning 22 Apr 2024 · 1 repository · arXiv:2404.14077
-
State-Separated SARSA: A Practical Sequential Decision-Making Algorithm with Recovering Rewards 18 Mar 2024 · 0 repositories · arXiv:2403.11520
-
Enhancing Classification Performance via Reinforcement Learning for Feature Selection 9 Mar 2024 · 0 repositories · arXiv:2403.05979
-
An Index Policy Based on Sarsa and Q-learning for Heterogeneous Smart Target Tracking 19 Feb 2024 · 0 repositories · arXiv:2402.12015
-
Using Reinforcement Learning to Optimize Responses in Care Processes: A Case Study on Aggression Incidents 2 Oct 2023 · 0 repositories · arXiv:2310.00981
-
Career Path Recommendations for Long-term Income Maximization: A Reinforcement Learning Approach 11 Sep 2023 · 0 repositories · arXiv:2309.05391
-
Exploring reinforcement learning techniques for discrete and continuous control tasks in the MuJoCo environment 20 Jul 2023 · 1 repository · arXiv:2307.11166
-
PCG-based Static Underground Garage Scenario Generation 8 Jul 2023 · 0 repositories · arXiv:2307.03988
-
Convergence of SARSA with linear function approximation: The random horizon case 7 Jun 2023 · 0 repositories · arXiv:2306.04548
-
On Modeling Network Slicing Communication Resources with SARSA Optimization 11 Jan 2023 · 0 repositories · arXiv:2301.04696
-
Analysis of Reinforcement Learning Schemes for Trajectory Optimization of an Aerial Radio Unit 18 Nov 2022 · 0 repositories · arXiv:2211.10524
-
SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions 5 Sep 2022 · 0 repositories · arXiv:2209.01876
-
Solving Royal Game of Ur Using Reinforcement Learning 23 Aug 2022 · 2 repositories · arXiv:2208.10669
-
On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios 19 Jul 2022 · 0 repositories · arXiv:2207.09372
-
Reinforced Lin-Kernighan-Helsgaun Algorithms for the Traveling Salesman Problems 8 Jul 2022 · 1 repository · arXiv:2207.03876
-
Does DQN Learn? 26 May 2022 · 0 repositories · arXiv:2205.13617
-
On the Convergence of SARSA with Linear Function Approximation 14 Feb 2022 · 0 repositories · arXiv:2202.06828
-
A study of first-passage time minimization via Q-learning in heated gridworlds 5 Oct 2021 · 0 repositories · arXiv:2110.02129
-
Enhancing Video Analytics Accuracy via Real-time Automated Camera Parameter Tuning 8 Jul 2021 · 0 repositories · arXiv:2107.03964
-
Reinforcement Learning for Physical Layer Communications 22 Jun 2021 · 1 repository · arXiv:2106.11595
-
A Provably-Efficient Model-Free Algorithm for Constrained Markov Decision Processes 3 Jun 2021 · 0 repositories · arXiv:2106.01577
-
Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning 3 Jan 2021 · 0 repositories · arXiv:2101.00687
-
Combining Reinforcement Learning with Lin-Kernighan-Helsgaun Algorithm for the Traveling Salesman Problem 8 Dec 2020 · 1 repository · arXiv:2012.04461
-
Obtain Employee Turnover Rate and Optimal Reduction Strategy Based On Neural Network and Reinforcement Learning 1 Dec 2020 · 0 repositories · arXiv:2012.00583
-
Solving The Lunar Lander Problem under Uncertainty using Reinforcement Learning 24 Nov 2020 · 2 repositories · arXiv:2011.11850
-
Chrome Dino Run using Reinforcement Learning 15 Aug 2020 · 0 repositories · arXiv:2008.06799
-
Using Reinforcement Learning to Herd a Robotic Swarm to a Target Distribution 29 Jun 2020 · 0 repositories · arXiv:2006.15807
-
Model-free Reinforcement Learning for Stochastic Stackelberg Security Games 24 May 2020 · 0 repositories · arXiv:2005.11853
-
FlapAI Bird: Training an Agent to Play Flappy Bird Using Reinforcement Learning Techniques 21 Mar 2020 · 2 repositories · arXiv:2003.09579
-
Enhancing the Monte Carlo Tree Search Algorithm for Video Game Testing 17 Mar 2020 · 0 repositories · arXiv:2003.07813
-
A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry 21 Jun 2019 · 1 repository · arXiv:1906.11286
-
Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environment 23 May 2019 · 0 repositories · arXiv:1905.09673
-
Design of Artificial Intelligence Agents for Games using Deep Reinforcement Learning 10 May 2019 · 0 repositories · arXiv:1905.04127
-
Finite-Sample Analysis for SARSA with Linear Function Approximation 6 Feb 2019 · 0 repositories · arXiv:1902.02234
-
Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target 22 Jan 2019 · 1 repository · arXiv:1901.07510
-
Recursive Sparse Pseudo-input Gaussian Process SARSA 17 Nov 2018 · 0 repositories · arXiv:1811.07201
-
The Concept of Criticality in Reinforcement Learning 16 Oct 2018 · 0 repositories · arXiv:1810.07254
-
Smoothed Action Value Functions for Learning Gaussian Policies 6 Mar 2018 · 0 repositories · arXiv:1803.02348
-
Reactive Reinforcement Learning in Asynchronous Environments 16 Feb 2018 · 0 repositories · arXiv:1802.06139
-
A Unified Approach for Multi-step Temporal-Difference Learning with Eligibility Traces in Reinforcement Learning 9 Feb 2018 · 0 repositories · arXiv:1802.03171
-
Learning Gaussian Policies from Smoothed Action Value Functions 1 Jan 2018 · 0 repositories
-
Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation 30 Nov 2017 · 0 repositories · arXiv:1711.11486
-
Double Q(σ) and Q(σ, λ): Unifying Reinforcement Learning Control Algorithms 5 Nov 2017 · 0 repositories · arXiv:1711.01569
-
A Comparison of Reinforcement Learning Techniques for Fuzzy Cloud Auto-Scaling 19 May 2017 · 0 repositories · arXiv:1705.07114
-
Multi-step Reinforcement Learning: A Unifying Algorithm 3 Mar 2017 · 0 repositories · arXiv:1703.01327
-
Online Transfer Learning in Reinforcement Learning Domains 2 Jul 2015 · 0 repositories · arXiv:1507.00436
-
Convergent Temporal-Difference Learning with Arbitrary Smooth Function Approximation 1 Dec 2009 · 0 repositories
Tasks archive 2025-07-28
20 shown of 38 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections