{"url":"/method/entropy-regularization","slug":"entropy-regularization","name":"Entropy Regularization","full_name":"Entropy Regularization","full_name_withheld":false,"description_markdown":"**Entropy Regularization** is a type of regularization used in [reinforcement learning](https://paperswithcode.com/methods/area/reinforcement-learning). For on-policy policy gradient based methods like [A3C](https://paperswithcode.com/method/a3c), the same mutual  reinforcement behaviour leads to a highly-peaked $\\pi\\left(a\\mid{s}\\right)$ towards a few actions or action sequences, since it is easier for the actor and critic to overoptimise to a small portion of the environment. To reduce this problem, entropy regularization adds an entropy term to the loss to promote action diversity:\r\n\r\n$$H(X) = -\\sum\\pi\\left(x\\right)\\log\\left(\\pi\\left(x\\right)\\right) $$\r\n\r\nImage Credit: Wikipedia","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1602.01783v2","title":"Asynchronous Methods for Deep Reinforcement Learning","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/ikostrikov/pytorch-a3c/blob/48d95844755e2c3e2c7e48bbd1a7141f7212b63f/train.py#L100","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":1128,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/viability-of-future-actions-robust-safety-in","title":"Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization","date":"2025-06-12","arxiv_id":"2506.10871","n_code_links":1,"syntology":null},{"paper":null,"title":"Ego-centric Learning of Communicative World Models for Autonomous Driving","date":"2025-06-09","arxiv_id":"2506.08149","n_code_links":0,"syntology":null},{"paper":null,"title":"Autonomous Vehicle Lateral Control Using Deep Reinforcement Learning with MPC-PID Demonstration","date":"2025-06-04","arxiv_id":"2506.04040","n_code_links":0,"syntology":null},{"paper":null,"title":"PPO in the Fisher-Rao geometry","date":"2025-06-04","arxiv_id":"2506.03757","n_code_links":0,"syntology":null},{"paper":"/paper/redrft-a-light-weight-benchmark-for","title":"RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming","date":"2025-06-04","arxiv_id":"2506.04302","n_code_links":1,"syntology":null},{"paper":null,"title":"Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective","date":"2025-06-03","arxiv_id":"2506.02553","n_code_links":0,"syntology":null},{"paper":null,"title":"Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening","date":"2025-06-03","arxiv_id":"2506.02355","n_code_links":0,"syntology":null},{"paper":"/paper/the-surprising-effectiveness-of-negative","title":"The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning","date":"2025-06-02","arxiv_id":"2506.01347","n_code_links":1,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":0}},{"paper":null,"title":"DriveMind: A Dual-VLM based Reinforcement Learning Framework for Autonomous Driving","date":"2025-06-01","arxiv_id":"2506.00819","n_code_links":0,"syntology":null},{"paper":null,"title":"Language-Guided Multi-Agent Learning in Simulations: A Unified Framework and Evaluation","date":"2025-06-01","arxiv_id":"2506.04251","n_code_links":0,"syntology":null},{"paper":"/paper/using-diffusion-ensembles-to-estimate","title":"Using Diffusion Ensembles to Estimate Uncertainty for End-to-End Autonomous Driving","date":"2025-05-31","arxiv_id":"2506.00560","n_code_links":0,"syntology":null},{"paper":"/paper/areal-a-large-scale-asynchronous","title":"AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning","date":"2025-05-30","arxiv_id":"2505.24298","n_code_links":1,"syntology":{"ran":0,"of":13,"unverified":13,"pointer_only":0}},{"paper":"/paper/composite-reward-design-in-ppo-driven","title":"Composite Reward Design in PPO-Driven Adaptive Filtering","date":"2025-05-29","arxiv_id":"2506.06323","n_code_links":1,"syntology":null},{"paper":"/paper/segment-policy-optimization-effective-segment","title":"Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models","date":"2025-05-29","arxiv_id":"2505.23564","n_code_links":1,"syntology":{"ran":7,"of":9,"unverified":2,"pointer_only":0}},{"paper":null,"title":"TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks","date":"2025-05-29","arxiv_id":"2505.23949","n_code_links":0,"syntology":null},{"paper":"/paper/r1-code-interpreter-training-llms-to-reason","title":"R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised and Reinforcement Learning","date":"2025-05-27","arxiv_id":"2505.21668","n_code_links":1,"syntology":null},{"paper":null,"title":"What Can RL Bring to VLA Generalization? An Empirical Study","date":"2025-05-26","arxiv_id":"2505.19789","n_code_links":0,"syntology":null},{"paper":"/paper/improving-value-estimation-critically","title":"Improving Value Estimation Critically Enhances Vanilla Policy Gradient","date":"2025-05-25","arxiv_id":"2505.19247","n_code_links":1,"syntology":null},{"paper":null,"title":"GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning","date":"2025-05-24","arxiv_id":"2505.18763","n_code_links":0,"syntology":null},{"paper":"/paper/a-robust-ppo-optimized-tabular-transformer","title":"A Robust PPO-optimized Tabular Transformer Framework for Intrusion Detection in Industrial IoT Systems","date":"2025-05-23","arxiv_id":"2505.18234","n_code_links":1,"syntology":null},{"paper":"/paper/shioenv-a-cli-behavior-capturing-environment","title":"ShIOEnv: A CLI Behavior-Capturing Environment Enabling Grammar-Guided Command Synthesis for Dataset Curation","date":"2025-05-23","arxiv_id":"2505.18374","n_code_links":1,"syntology":null},{"paper":"/paper/the-cell-must-go-on-agar-io-for-continual","title":"The Cell Must Go On: Agar.io for Continual Reinforcement Learning","date":"2025-05-23","arxiv_id":"2505.18347","n_code_links":1,"syntology":null},{"paper":"/paper/raw2drive-reinforcement-learning-with-aligned","title":"Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)","date":"2025-05-22","arxiv_id":"2505.16394","n_code_links":0,"syntology":null},{"paper":null,"title":"AM-PPO: (Advantage) Alpha-Modulation with Proximal Policy Optimization","date":"2025-05-21","arxiv_id":"2505.15514","n_code_links":0,"syntology":null},{"paper":null,"title":"HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving","date":"2025-05-21","arxiv_id":"2505.15793","n_code_links":0,"syntology":null},{"paper":"/paper/ipad-iterative-proposal-centric-end-to-end","title":"iPad: Iterative Proposal-centric End-to-End Autonomous Driving","date":"2025-05-21","arxiv_id":"2505.15111","n_code_links":1,"syntology":null},{"paper":"/paper/trajectory-bellman-residual-minimization-a","title":"Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning","date":"2025-05-21","arxiv_id":"2505.15311","n_code_links":0,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":0}},{"paper":null,"title":"Interpretable Reinforcement Learning for Load Balancing using Kolmogorov-Arnold Networks","date":"2025-05-20","arxiv_id":"2505.14459","n_code_links":0,"syntology":null},{"paper":null,"title":"KIPPO: Koopman-Inspired Proximal Policy Optimization","date":"2025-05-20","arxiv_id":"2505.14566","n_code_links":0,"syntology":null},{"paper":null,"title":"PEER pressure: Model-to-Model Regularization for Single Source Domain Generalization","date":"2025-05-19","arxiv_id":"2505.12745","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":376},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":328},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":301},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":248},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":183},{"task":"/task/autonomous-vehicles","name":"Autonomous Vehicles","papers":80},{"task":"/task/decision-making","name":"Decision Making","papers":70},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":60},{"task":"/task/object-detection","name":"Object Detection","papers":54},{"task":"/task/object-detection-1","name":"object-detection","papers":50},{"task":"/task/continuous-control","name":"Continuous Control","papers":47},{"task":"/task/mujoco","name":"MuJoCo","papers":44},{"task":"/task/continuous-control","name":"continuous-control","papers":43},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":40},{"task":"/task/q-learning","name":"Q-Learning","papers":38},{"task":"/task/language-modelling","name":"Language Modelling","papers":34},{"task":"/task/atari-games","name":"Atari Games","papers":31},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":30},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":26},{"task":"/task/language-modeling","name":"Language Modeling","papers":25}],"tasks_shown":20,"n_tasks":404,"usage_by_year":[{"year":"2016","papers":4},{"year":"2017","papers":17},{"year":"2018","papers":34},{"year":"2019","papers":72},{"year":"2020","papers":123},{"year":"2021","papers":133},{"year":"2022","papers":168},{"year":"2023","papers":174},{"year":"2024","papers":260},{"year":"2025","papers":143}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/entropy-regularization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}