Methods › Reinforcement Learning › Policy Gradient Methods › Soft Actor Critic
Soft Actor Critic
Introduced by Tuomas Haarnoja et al. in Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Soft Actor Critic, or SAC, is an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy. That is, to succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. SAC combines off-policy updates with a stable stochastic actor-critic formulation.
The SAC objective has a number of advantages. First, the policy is incentivized to explore more widely, while giving up on clearly unpromising avenues. Second, the policy can capture multiple modes of near-optimal behavior. In problem settings where multiple actions seem equally attractive, the policy will commit equal probability mass to those actions. Lastly, the authors present evidence that it improves learning speed over state-of-art methods that optimize the conventional RL objective function.
In Syntology for a different name, 'soft actor-critic': repository known, no verified implementation (declared_code; repositories: 5 listed of 86 in the graph, strongest link: official). The lookup matched a different name in Syntology's graph, 'soft actor-critic' (resolution: token_match); this answer is about that name. This answer is about arXiv:1801.01290, the paper the archive credits with this method.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- SOFT · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='SOFT') - Soft MoE · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Soft MoE') - soft softmax-triplet loss · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='soft softmax-triplet loss') - MODEL-AUGMENTED ACTOR-CRITIC · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='MODEL-AUGMENTED ACTOR-CRITIC') - Critic Regularized Regression · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Critic Regularized Regression')
Origin paper of this answer, arXiv:1801.01290 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 91 ran (of which 61 constructed an object rather than computing a result; 73 with no instrument failure: 3 honoured, 1 violated, 69 with no contract checked; 18 where Syntology's instrument failed) · 57 unverified (of 148 harvested samples) · 66 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 58 · with a code link: 22 · where Syntology ran a sample: 9 (8 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 58 tagged: 8 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 58 of 58, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss 14 Apr 2025 · 0 repositories · arXiv:2504.09929
-
Closing the Intent-to-Behavior Gap via Fulfillment Priority Logic 4 Mar 2025 · 0 repositories · arXiv:2503.05818
-
IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic 27 Feb 2025 · 0 repositories · arXiv:2502.19859
-
Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning 29 Jan 2025 · 1 repository · arXiv:2501.17827Syntology official (archive's flag): 6 ran · 6 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples)
-
Reinforcement Learning Controlled Adaptive PSO for Task Offloading in IIoT Edge Computing 25 Jan 2025 · 1 repository · arXiv:2501.15203
-
Average Reward Reinforcement Learning for Wireless Radio Resource Management 12 Jan 2025 · 0 repositories · arXiv:2501.06700
-
Learn 2 Rage: Experiencing The Emotional Roller Coaster That Is Reinforcement Learning 24 Oct 2024 · 0 repositories · arXiv:2410.18462
-
Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control 19 Oct 2024 · 0 repositories · arXiv:2410.15188
-
Solving The Dynamic Volatility Fitting Problem: A Deep Reinforcement Learning Approach 15 Oct 2024 · 0 repositories · arXiv:2410.11789
-
Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment 12 Jul 2024 · 1 repository · arXiv:2407.08932
-
Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning? 10 Jul 2024 · 0 repositories · arXiv:2407.07364
-
Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigation 8 Jul 2024 · 0 repositories · arXiv:2407.06317
-
A fast balance optimization approach for charging enhancement of lithium-ion battery packs through deep reinforcement learning 24 Apr 2024 · 1 repository
-
Imitation Game: A Model-based and Imitation Learning Deep Reinforcement Learning Hybrid 2 Apr 2024 · 0 repositories · arXiv:2404.01794
-
K-percent Evaluation for Lifelong RL 2 Apr 2024 · 0 repositories · arXiv:2404.02113
-
Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle 5 Jan 2024 · 0 repositories · arXiv:2401.02903
-
Dynamic Fairness-Aware Spectrum Auction for Enhanced Licensed Shared Access in 6G Networks 20 Dec 2023 · 0 repositories · arXiv:2312.12867
-
On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning 13 Dec 2023 · 0 repositories · arXiv:2312.07917
-
Joint Sensing and Communication Optimization in Target-Mounted STARS-Assisted Vehicular Networks: A MADRL Approach 17 Nov 2023 · 0 repositories · arXiv:2311.10352
-
Belief Projection-Based Reinforcement Learning for Environments with Delayed Feedback 21 Sep 2023 · 1 repository
-
Hybrid of representation learning and reinforcement learning for dynamic and complex robotic motion planning 7 Sep 2023 · 0 repositories · arXiv:2309.03758
-
A Safe Deep Reinforcement Learning Approach for Energy Efficient Federated Learning in Wireless Communication Networks 21 Aug 2023 · 0 repositories · arXiv:2308.10664
-
Hint assisted reinforcement learning: an application in radio astronomy 10 Jan 2023 · 1 repository · arXiv:2301.03933
-
Efficient Exploration in Resource-Restricted Reinforcement Learning 14 Dec 2022 · 0 repositories · arXiv:2212.06988
-
RL-Based Guidance in Outpatient Hysteroscopy Training: A Feasibility Study 26 Nov 2022 · 0 repositories · arXiv:2211.14541
-
A Deep Reinforcement Learning-Based Charging Scheduling Approach with Augmented Lagrangian for Electric Vehicle 20 Sep 2022 · 0 repositories · arXiv:2209.09772
-
Plug-and-Play Model-Agnostic Counterfactual Policy Synthesis for Deep Reinforcement Learning based Recommendation 10 Aug 2022 · 0 repositories · arXiv:2208.05142
-
Imitation Learning by State-Only Distribution Matching 9 Feb 2022 · 1 repository · arXiv:2202.04332Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning 14 Jan 2022 · 1 repository · arXiv:2201.05599
-
Actor Loss of Soft Actor Critic Explained 31 Dec 2021 · 0 repositories · arXiv:2112.15568
-
Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization 8 Dec 2021 · 0 repositories · arXiv:2112.04084
-
Mixture-of-Variational-Experts for Continual Learning 25 Oct 2021 · 1 repository · arXiv:2110.12667
-
Risk Conditioned Neural Motion Planning 4 Aug 2021 · 1 repository · arXiv:2108.01851
-
Context-Based Soft Actor Critic for Environments with Non-stationary Dynamics 7 May 2021 · 1 repository · arXiv:2105.03310
-
End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learning 26 Apr 2021 · 1 repository · arXiv:2104.12842
-
Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm 30 Jan 2021 · 0 repositories · arXiv:2102.00168
-
Factored Action Spaces in Deep Reinforcement Learning 1 Jan 2021 · 0 repositories
-
Learning from Simulation, Racing in Reality 26 Nov 2020 · 0 repositories · arXiv:2011.13332
-
Deep RL With Information Constrained Policies: Generalization in Continuous Control 9 Oct 2020 · 0 repositories · arXiv:2010.04646
-
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL 21 Jul 2020 · 0 repositories · arXiv:2007.11091
-
Developing cooperative policies for multi-stage tasks 1 Jul 2020 · 0 repositories · arXiv:2007.00203
-
Experience Replay with Likelihood-free Importance Weights 23 Jun 2020 · 1 repository · arXiv:2006.13169Syntology 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
Smooth Exploration for Robotic Reinforcement Learning 12 May 2020 · 4 repositories · arXiv:2005.05719Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
DSAC: Distributional Soft Actor Critic for Risk-Sensitive Reinforcement Learning 30 Apr 2020 · 0 repositories · arXiv:2004.14547
-
Reinforcement Learning with Augmented Data 30 Apr 2020 · 2 repositories · arXiv:2004.14990Syntology 21 ran (of which 8 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 12 where Syntology's instrument failed) · 3 unverified (of 24 harvested samples) · 21 pointer-only (licence)
-
Adaptive Experience Selection for Policy Gradient 17 Feb 2020 · 0 repositories · arXiv:2002.06946
-
Cooperative Highway Work Zone Merge Control based on Reinforcement Learning in A Connected and Automated Environment 21 Jan 2020 · 0 repositories · arXiv:2001.08581
-
Discriminator Soft Actor Critic without Extrinsic Rewards 19 Jan 2020 · 1 repository · arXiv:2001.06808
-
SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning 28 Dec 2019 · 1 repository · arXiv:1912.12482Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples)
-
Policy Optimization Reinforcement Learning with Entropy Regularization 2 Dec 2019 · 0 repositories · arXiv:1912.01557
-
Better Exploration with Optimistic Actor Critic 1 Dec 2019 · 1 repository
-
Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning 15 Nov 2019 · 0 repositories · arXiv:1911.06832
-
Better Exploration with Optimistic Actor-Critic 28 Oct 2019 · 0 repositories · arXiv:1910.12807
-
Striving for Simplicity and Performance in Off-Policy DRL: Output Normalization and Non-Uniform Sampling 5 Oct 2019 · 3 repositories · arXiv:1910.02208
-
Modified Actor-Critics 2 Jul 2019 · 0 repositories · arXiv:1907.01298
-
Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies 6 Jun 2019 · 1 repository · arXiv:1906.02771Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 3 pointer-only (licence)
-
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models 30 May 2018 · 9 repositories · arXiv:1805.12114Syntology community repositories only · 12 ran (of which 5 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 7 unverified (of 19 harvested samples) · 17 pointer-only (licence)
-
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor 4 Jan 2018 · 86 repositories · arXiv:1801.01290Syntology community repositories only · 91 ran (of which 61 constructed an object rather than computing a result; 73 with no instrument failure: 3 honoured, 1 violated, 69 with no contract checked; 18 where Syntology's instrument failed) · 57 unverified (of 148 harvested samples) · 66 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 55 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections