Papers › Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

3 Jul 2015arXiv:1507.00814archive 2025-07-28

Bradly C. Stadie, Sergey Levine, Pieter Abbeel

Achieving efficient and scalable exploration in complex domains poses a major challenge in reinforcement learning. While Bayesian and PAC-MDP approaches to the exploration problem offer strong formal guarantees, they are often impractical in higher dimensions due to their reliance on enumerating the state-action space. Hence, exploration in complex domains is often performed with simple epsilon-greedy methods. In this paper, we consider the challenging Atari games domain, which requires processing raw pixel inputs and delayed rewards. We evaluate several more sophisticated exploration strategies, including Thompson sampling and Boltzman exploration, and propose a new exploration method based on assigning exploration bonuses from a concurrently learned model of the system dynamics. By parameterizing our learned model with a neural network, we are able to develop a scalable and efficient approach to exploration bonuses that can be applied to tasks with complex, high-dimensional state spaces. In the Atari domain, our method provides the most consistent improvement across a range of games that pose a major challenge for prior methods. In addition to raw game-scores, we also develop an AUC-100 metric for the Atari Learning domain to evaluate the impact of exploration on this benchmark.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

CoffeeddCat/Multiagent_chainMDP mentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Atari Games Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling reinforcement-learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Atari Games	Atari 2600 Freeway	MP-EB	Score	27.0	#42 of 59	Archive leaderboard	report
Atari Games	Atari 2600 Frostbite	MP-EB	Score	507.0	#38 of 53	Archive leaderboard	report
Atari Games	Atari 2600 Montezuma's Revenge	MP-EB	Score	142	#25 of 50	Archive leaderboard	report
Atari Games	Atari 2600 Q*Bert	MP-EB	Score	15805	#24 of 57	Archive leaderboard	report
Atari Games	Atari 2600 Venture	MP-EB	Score	0.0	#49 of 55	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections