Papers › GDI: Rethinking What Makes Reinforcement Learning Different from Supervised Learning

GDI: Rethinking What Makes Reinforcement Learning Different from Supervised Learning

24 Nov 2021AAAI Workshop ML4OR-22 2022 2archive 2025-07-28

Anonymous

Deep Q Network (DQN) firstly kicked the door of deep reinforcement learning (DRL) via combining deep learning (DL) with reinforcement learning (RL), which has noticed that the distribution of the acquired data would change during the training process. DQN found this property might cause instability for training, so it proposed effective methods to handle the downside of the property. Instead of focusing on the unfavorable aspects, we find it critical for RL to ease the gap between the estimated data distribution and the ground truth data distribution while supervised learning (SL) fails to do so. From this new perspective, we extend the basic paradigm of RL called the Generalized Policy Iteration (GPI) into a more generalized version, which is called the Generalized Data Distribution Iteration (GDI). We see massive RL algorithms and techniques can be unified into the GDI paradigm, which can be considered as one of the special cases of GDI. We provide theoretical proof of why GDI is better than GPI and how it works. Several practical algorithms based on GDI have been proposed to verify its effectiveness and extensiveness. Empirical experiments prove our state-of the-art (SOTA) performance on Arcade Learning Environment (ALE), wherein our algorithm has achieved 9620.98% mean human normalized score (HNS), 1146.39% median HNS and 22 human world record breakthroughs (HWRB) using only 200M training frames. Our work aims to lead the RL research to step into the journey of conquering the human world records and seek real superhuman agents on both performance and efficiency.

Paper PDF Conference PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Atari Games Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)reinforcement-learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Atari Games	Atari 2600 Alien	GDI-H3(1B frames)	Score	279700	#3 of 49	Archive leaderboard	report
Atari Games	Atari 2600 Centipede	GDI-H3(1B frames)	Score	1359533	#2 of 45	Archive leaderboard	report
Atari Games	Atari 2600 Kung-Fu Master	GDI-H3 (200M)	Score	1666000	#2 of 43	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Convolution DQN Dense Connections Q-Learning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections