Papers › Deep Reinforcement Learning with Double Q-learning

Deep Reinforcement Learning with Double Q-learning

22 Sep 2015arXiv:1509.06461archive 2025-07-28

Hado van Hasselt, Arthur Guez, David Silver

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.

PaperPDFCodeCode Syntology ranMethod as code

In Syntology View this paper on Syntology, its page in Syntology's graph. That page adds which of its functions are also present in other papers' code Syntology holds, with dates, and its dated run record as JSON and a README badge. It also lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

Syntology 106 code samples harvested from 37 repositories linked to this paper: 56 ran (of which 38 constructed an object rather than computing a result; 55 with no instrument failure: 0 honoured, 0 violated, 55 with no contract checked; 1 where Syntology's instrument failed) · 50 unverified. Where Syntology's instrument failed: 1 where the contract check could not run; failures of Syntology's instrument, not of the code.

By repository: community (archive-listed): 106 samples from 37 repositories, 56 ran (55 with no instrument failure; 1 where Syntology's instrument failed). The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

No sample is linked to this paper from 21 of its repositories that Syntology harvested for other papers:

Levels written between 2026-09-22 and 2026-09-24 (UTC): the dates Syntology's graph wrote each sample's verification level, not when the code executed.

97 repositories listed; official and paper-mentioned ones first.

1jsingh/rl_navigation mentioned on GitHubpytorchSyntology: 5 samples, 3 ran (0 instrument-failed) · 2 unverified report
Adrelf/DRL-navigation mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (0 instrument-failed) · 0 unverified report
Codernauti/Exploration-of-DQN-in-CartPole-Environment mentioned on GitHubSyntology: not harvested report
Curt-Park/rainbow-is-all-you-need mentioned on GitHubMITSyntology: not harvested report
Denbergvanthijs/imbDRL mentioned on GitHubtfApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Deep Reinforcement Learning for Imbalanced Classification (arXiv:1901.01379; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report
FaboNo/DRLND mentioned on GitHubpytorchGPL-3.0Syntology: not harvested report
HussonnoisMaxence/RL_Algorithms mentioned on GitHubpytorchSyntology: 3 samples, 1 ran (0 instrument-failed) · 2 unverified report
JonasRSV/DQN mentioned on GitHubtfSyntology: not harvested report
JonasRSV/DQNTensorflow mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified); nothing checks that they implement this paper's method. report
JustinStitt/acrobotDDQN mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
KelvinYang0320/deepbots-panda mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Continuous control with deep reinforcement learning (arXiv:1509.02971; that paper's own run record: 163 ran (of which 126 constructed an object rather than computing a result; 152 with no instrument failure: 3 honoured, 0 violated, 149 with no contract checked; 11 where Syntology's instrument failed) · 143 unverified); nothing checks that they implement this paper's method. report
MEOWMEOW114/nd893-p1-navigation-banana mentioned on GitHubpytorchSyntology: 4 samples, 3 ran (0 instrument-failed) · 1 unverified report
MOVzeroOne/DQN mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Dueling Network Architectures for Deep Reinforcement Learning (arXiv:1511.06581; that paper's own run record: 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report
NikolausBerl/Udacity_DRLN_Navigation_Project mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
OMS1996/Carla_The_RL_Self-Driving-Car mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
PeterJochem/Deep_RL mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
PeterJochem/Double_Deep_QLearning mentioned on GitHubtfSyntology: not harvested report
Rabrg/dqn mentioned on GitHubpytorchSyntology: not harvested report
RandyDeng/gym_connect4 mentioned on GitHubGPL-3.0Syntology: not harvested report
Roman-Kozachek/TradeBot mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
SayhoKim/tetrisRL mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified) and Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
YuansongFeng/MadMario mentioned on GitHubpytorchSyntology: not harvested report
ZainRaza14/deepRL mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Continuous control with deep reinforcement learning (arXiv:1509.02971; that paper's own run record: 163 ran (of which 126 constructed an object rather than computing a result; 152 with no instrument failure: 3 honoured, 0 violated, 149 with no contract checked; 11 where Syntology's instrument failed) · 143 unverified); nothing checks that they implement this paper's method. report
aman-khurana/deep-q-learning mentioned on GitHubSyntology: not harvested report
amirmirzaei79/CartPole-DQN-And-DDQN mentioned on GitHubpytorchSyntology: not harvested report
anh-nn01/Lunar-Lander-Double-Deep-Q-Networks mentioned on GitHubtfMITSyntology: not harvested report
atavakol/action-branching-agents mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
austinsilveria/Banana-Collection-DQN mentioned on GitHubpytorchSyntology: not harvested report
botforge/simplementation mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (0 instrument-failed) · 1 unverified report
chainer/chainerrl mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ChainerRL: A Deep Reinforcement Learning Library (arXiv:1912.03905; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
cocolico14/N-step-Dueling-DDQN-PER-Pacman mentioned on GitHubSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
cove9988/TradingGym mentioned on GitHubSyntology: not harvested report
daviddcho/supermario mentioned on GitHubpytorchSyntology: 5 samples, 4 ran (0 instrument-failed) · 1 unverified report
dxyang/DQN_pytorch mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Dueling Network Architectures for Deep Reinforcement Learning (arXiv:1511.06581; that paper's own run record: 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report
facebookresearch/ReAgent mentioned on GitHubpytorchSyntology: not harvested report
fengsterooni/dql mentioned on GitHubpytorchSyntology: 3 samples, 1 ran (0 instrument-failed) · 2 unverified report
guillaumeboniface/bananaland mentioned on GitHubpytorchSyntology: 4 samples, 3 ran (0 instrument-failed) · 1 unverified report
gznyyb/deep_reinforcement_learning_Pong mentioned on GitHubtfSyntology: not harvested report
hamishs/JAX-RL mentioned on GitHubjaxSyntology: 4 samples, 3 ran (0 instrument-failed) · 1 unverified report
hemilpanchiwala/Dueling-Network-Architectures mentioned on GitHubpytorchSyntology: 3 samples, 0 ran · 3 unverified report
hemilpanchiwala/Dueling_Network_Architectures mentioned on GitHubpytorchSyntology: not harvested report
ianlimle/ItsMeMario mentioned on GitHubpytorchnot reachable when probed 2026-09-18 — repositories for recent papers often appear after camera-readySyntology: not harvested report
ifestus/rl mentioned on GitHubtfSyntology: 3 samples, 0 ran · 3 unverified report
jadag/DDQN_mario mentioned on GitHubtfSyntology: not harvested report
jeffery1236/Atari_DoubleDeepQNetwork mentioned on GitHubpytorchSyntology: 3 samples, 0 ran · 3 unverified report
jezzarax/drlnd_p1_navigation mentioned on GitHubpytorchSyntology: 2 samples, 2 ran (0 instrument-failed) · 0 unverified report
jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning mentioned on GitHubtfMITSyntology: 2 samples, 0 ran · 2 unverified report
jvoynow/DQN-analysis-with-2048 mentioned on GitHubSyntology: not harvested report
kmdanielduan/DQN_Family_PyTorch mentioned on GitHubpytorchSyntology: 5 samples, 3 ran (0 instrument-failed) · 2 unverified report
kochlisGit/autonomous-vehicles-agent mentioned on GitHubtfSyntology: not harvested report
kshitij-ingale/Reinforcement-Learning mentioned on GitHubtfSyntology: not harvested report
lbenbaccar/Deep-Reinforcement-Learning-with-Double-Q-Learning mentioned on GitHubSyntology: not harvested report
marload/DeepRL-TensorFlow2 mentioned on GitHubtfSyntology: 3 samples, 1 ran (0 instrument-failed) · 2 unverified report
matthewsparr/Deep-Zork mentioned on GitHubSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
microsoft/med-deadend mentioned on GitHubpytorchSyntology: 5 samples, 3 ran (0 instrument-failed) · 2 unverified report
mightypirate1/DRL-Tetris mentioned on GitHubtfGPL-3.0Syntology: not harvested report
mindspore-courses/Deep-Reinforcement-Learning-Algorithms-with-MindSpore mentioned on GitHubmindsporeSyntology: not harvested report
mindspore-courses/Rainbow-MindSpore mentioned on GitHubmindsporeSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
moduIo/Deep-Q-network mentioned on GitHubSyntology: not harvested report
mohit8935/Deep-Q-Learning-Paper mentioned on GitHubpytorchSyntology: 2 samples, 2 ran (0 instrument-failed) · 0 unverified report
molomono/CartPole_Optimized_DDQN mentioned on GitHubtfSyntology: not harvested report
nbopardi/smb mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
near32/regym mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
paintception/Deep-Quality-Value-Family mentioned on GitHubSyntology: not harvested report
paintception/Deep-Quality-Value-Family- mentioned on GitHubSyntology: not harvested report
pathway/alphaxos mentioned on GitHubSyntology: not harvested report
philtabor/Deep-Q-Learning-Paper-To-Code mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (0 instrument-failed) · 0 unverified report
puppetect/TradingBot-tensorflow mentioned on GitHubtfSyntology: not harvested report
rybread1/DeepRlTrex mentioned on GitHubtfnot reachable when probed 2026-09-18 — repositories for recent papers often appear after camera-readySyntology: not harvested report
rybread1/deep-rl-trex mentioned on GitHubtfnot reachable when probed 2026-09-18 — repositories for recent papers often appear after camera-readySyntology: not harvested report
seacevedo/ReinforcementLearningProjects mentioned on GitHubpytorchSyntology: 2 samples, 2 ran (0 instrument-failed) · 0 unverified report
shashwatsaxena571/DRL-navigation mentioned on GitHubpytorchnot reachable when probed 2026-09-18 — repositories for recent papers often appear after camera-readySyntology: not harvested report
shehrum/RL_Navigation mentioned on GitHubpytorchSyntology: 2 samples, 0 ran · 2 unverified report
ssainz/reinforcement_learning_algorithms mentioned on GitHubpytorchSyntology: 3 samples, 2 ran (0 instrument-failed) · 1 unverified report
tensorlayer/RLzoo mentioned on GitHubtfSyntology: 2 samples, 0 ran · 2 unverified report
tensorpack/tensorpack mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified), Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (arXiv:1511.06434; that paper's own run record: 128 ran (of which 61 constructed an object rather than computing a result; 111 with no instrument failure: 3 honoured, 0 violated, 108 with no contract checked; 17 where Syntology's instrument failed) · 91 unverified) and Improved Training of Wasserstein GANs (arXiv:1704.00028; that paper's own run record: 27 ran (of which 10 constructed an object rather than computing a result; 19 with no instrument failure: 1 honoured, 0 violated, 18 with no contract checked; 8 where Syntology's instrument failed) · 22 unverified); nothing checks that they implement this paper's method. report
tkcoding/DeepRL mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
tkcoding/Stock_DRL mentioned on GitHubpytorchSyntology: not harvested report
utarumo/RL_implementation mentioned on GitHubtfSyntology: 3 samples, 2 ran (0 instrument-failed) · 1 unverified report
wmol4/Pytorch_DDQN_Unity_Navigation mentioned on GitHubpytorchSyntology: not harvested report
wtingda/DeepRLBreakout mentioned on GitHubtfMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Asynchronous Methods for Deep Reinforcement Learning (arXiv:1602.01783; that paper's own run record: 60 ran (of which 20 constructed an object rather than computing a result; 50 with no instrument failure: 2 honoured, 1 violated, 47 with no contract checked; 10 where Syntology's instrument failed) · 35 unverified); nothing checks that they implement this paper's method. report
xgfelicia/Reinforcement-Learning mentioned on GitHubpytorchSyntology: 2 samples, 2 ran (0 instrument-failed) · 0 unverified report
xtma/simple-pytorch-rl mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Continuous control with deep reinforcement learning (arXiv:1509.02971; that paper's own run record: 163 ran (of which 126 constructed an object rather than computing a result; 152 with no instrument failure: 3 honoured, 0 violated, 149 with no contract checked; 11 where Syntology's instrument failed) · 143 unverified) and Proximal Policy Optimization Algorithms (arXiv:1707.06347; that paper's own run record: 99 ran (of which 53 constructed an object rather than computing a result; 71 with no instrument failure: 7 honoured, 2 violated, 62 with no contract checked; 28 where Syntology's instrument failed) · 77 unverified); nothing checks that they implement this paper's method. report
yaxinchen666/dce_pricingRL mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified); nothing checks that they implement this paper's method. report
yukezhu/tensorflow-reinforce mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
yzheng51/rl-dino-run mentioned on GitHubpytorchSyntology: 7 samples, 6 ran (0 instrument-failed) · 1 unverified report
zhengant/dqn_reversi mentioned on GitHubtfSyntology: not harvested report
MaximeVandegar/Papers-in-100-Lines-of-Code pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified), Generative Adversarial Networks (arXiv:1406.2661; that paper's own run record: 46 ran (of which 0 constructed an object rather than computing a result; 28 with no instrument failure: 4 honoured, 1 violated, 23 with no contract checked; 18 where Syntology's instrument failed) · 9 unverified), NICE: Non-linear Independent Components Estimation (arXiv:1410.8516; that paper's own run record: 24 ran (of which 16 constructed an object rather than computing a result; 19 with no instrument failure: 0 honoured, 0 violated, 19 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified) and 30 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
OscarHuangWind/Preference-Guided-DQN-Atari pytorchSyntology: 5 samples, 4 ran (0 instrument-failed) · 1 unverified report
chandar-lab/RLHive pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified) and Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
deepmind/rlax jaxSyntology: 1 sample, 1 ran (1 instrument-failed) · 0 unverified report
hill-a/stable-baselines tfSyntology: 7 samples, 0 ran · 7 unverified report
labmlai/annotated_deep_learning_paper_implementations pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified), Adam: A Method for Stochastic Optimization (arXiv:1412.6980; that paper's own run record: 27 ran (of which 7 constructed an object rather than computing a result; 23 with no instrument failure: 4 honoured, 2 violated, 17 with no contract checked; 4 where Syntology's instrument failed) · 10 unverified), Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (arXiv:1502.03167; that paper's own run record: 19 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified) and 12 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
opendilab/DI-engine pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Hindsight Experience Replay (arXiv:1707.01495; that paper's own run record: 16 ran (of which 10 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified), Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (arXiv:1901.02860; that paper's own run record: 67 ran (of which 37 constructed an object rather than computing a result; 49 with no instrument failure: 4 honoured, 1 violated, 44 with no contract checked; 18 where Syntology's instrument failed) · 76 unverified), Decision Transformer: Reinforcement Learning via Sequence Modeling (arXiv:2106.01345; that paper's own run record: 17 ran (of which 10 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
toni-sm/skrl jaxMITSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

106 samples harvested: 56 ran (of which 38 constructed an object rather than computing a result; 55 with no instrument failure: 0 honoured, 0 violated, 55 with no contract checked; 1 where Syntology's instrument failed) · 50 unverified. “Where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Syntology's record for this page has not changed since , the first build that kept a record date for it; when this build read Syntology's graph is in the build record. That date is not when the samples ran.

1ran; the contract check could not run
55ran
50unverified

Licence: 57 of the 106 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 37 repositories linked to this paper, official or community; the list below is grouped by repository, each group headed by its name and what it is to the paper. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. A line beginning “fixture:” under a sample that ran gives the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right. “Level recorded” gives the UTC date when Syntology's graph wrote the sample's verification level, not when the code executed; where the level was written before Syntology's graph kept that date, the row says “level recorded, date not recorded”. For 5 samples that ran, the graph records a signature this page does not show: for a class the fixture built without calling, the harness called the object with inputs it made up itself, so the recorded signature is the harness's probe, not the fixture's arguments. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself. “Our draft was wrong” and “the contract check could not run” are failures of Syntology's instrument, not of the code: the sample ran on Syntology's synthesized input, and then the drafted contract did not describe it, or the check itself raised an error (RAISES), lacked a dependency (DEP_MISSING) or ran out of time (TIMEOUT); hover the label for which. A metamorphic tier, where shown, is evidence of the output's shape, not that the sample implements the method.

Each sample ends with its code_sha256. A sample's code_sha256 is Syntology's identity for that exact code. It is printed in full (16 hexadecimal characters for every sample in this build), and get_code(code_sha256="…") takes it exactly as printed. An agent calls it on Syntology's MCP service (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples are grouped by repository: a repository marked official first, then by how directly the repository is tied to the paper, then in the archive's order; never by how many ran. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of six reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar); this paper's copy was not located, so its licence is not recorded; or this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not. Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

A function or class name links to its page when the name occurs in the harvested code of at least 5 papers; the name is a string, not an identity. A census tag appears when Syntology's census ran implementations with that name on one shared input per bucket: it gives how many were compared, from how many papers, and how many distinct outputs they produced, and names this sample's bucket when this sample was one of them.

1jsingh/rl_navigation community (archive-listed) 5 samples, 3 ran (0 instrument-failed) · 2 unverified
FixedSize_BinarySearchTree agents/dqn_agent.py ran MIT (permissive) · 42487fb842418ae5 · report fixture: returned FixedSize_BinarySearchTree · level recorded 2026-09-22
Node in 32 papers agents/dqn_agent.py ran MIT (permissive) · 896578e9d4f08299 · report fixture: returned Node · level recorded 2026-09-22
Qnetwork agents/dqn_agent.py ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · 286e69c369a502d4 · report fixture: returned Qnetwork · level recorded 2026-09-22
Agent in 42 papers agents/dqn_agent.py unverified MIT (permissive) · f370c556306eb621 · report
ReplayBuffer in 33 papers agents/dqn_agent.py unverified MIT (permissive) · f07ed06c9c4c4e7c · report
Adrelf/DRL-navigation community (archive-listed) 3 samples, 3 ran (0 instrument-failed) · 0 unverified
Agent in 42 papers brain/dqn_agent.py ran no licence file found · pointer only · 3836a1d32ca55617 · report fixture: returned Agent · level recorded 2026-09-22
QNetwork in 11 papers brain/dqn_agent.py ran no licence file found · pointer only · 01d2de07d88f7c3a · report fixture: signature state:Tensor/2/float · returned Tensor · level recorded 2026-09-24
ReplayBuffer in 33 papers brain/dqn_agent.py ran no licence file found · pointer only · 0e225401249db6d9 · report fixture: returned ReplayBuffer · level recorded 2026-09-22
HussonnoisMaxence/RL_Algorithms community (archive-listed) 3 samples, 1 ran (0 instrument-failed) · 2 unverified
Network in 40 papers QLearning/D2QN/D2QN.py ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · 01fc22f5d7228bdb · report fixture: signature x:Tensor/2/float · returned Tensor · level recorded 2026-09-22
Agent in 42 papers QLearning/D2QN/D2QN.py unverified no licence file found · pointer only · 7802ec0bd30c8512 · report
ReplayMemory in 11 papers QLearning/D2QN/D2QN.py unverified no licence file found · pointer only · fd50d6e813cead15 · report
JustinStitt/acrobotDDQN community (archive-listed) 1 sample, 0 ran · 1 unverified
DoubleDQN DDQN.py unverified no licence file found · pointer only · f7926a68b7bdd88d · report
MEOWMEOW114/nd893-p1-navigation-banana community (archive-listed) 4 samples, 3 ran (0 instrument-failed) · 1 unverified
DDQNAgent dqn/double_dqn_agent.py ran fingerprinted no licence file found · pointer only · 00919eeae2ebc39b · report fixture: signature state:ndarray/1/float, eps:float/0/float · returned int64 · level recorded 2026-09-24
QNetwork in 11 papers dqn/double_dqn_agent.py ran fingerprinted no licence file found · pointer only · f9462232ec862c47 · report fixture: signature state:Tensor/2/float · returned Tensor · level recorded 2026-09-24
ReplayBuffer in 33 papers dqn/double_dqn_agent.py ran no licence file found · pointer only · dcd7958b0ad16526 · report fixture: returned ReplayBuffer · level recorded 2026-09-22
DQNAgent dqn/double_dqn_agent.py unverified no licence file found · pointer only · b9e4d950b7bc9105 · report
NikolausBerl/Udacity_DRLN_Navigation_Project community (archive-listed) 1 sample, 0 ran · 1 unverified
Agent in 42 papers dqn_agent.py unverified no licence file found · pointer only · 52999846a5c49f96 · report
PeterJochem/Deep_RL community (archive-listed) 1 sample, 0 ran · 1 unverified
Agent in 42 papers DDQN/BBreaker/myAgent.py unverified no licence file found · pointer only · a812e436e77634de · report
Roman-Kozachek/TradeBot community (archive-listed) 1 sample, 0 ran · 1 unverified
DQNAgent classes/agent.py unverified no licence file found · pointer only · 9c6c6632e740cb42 · report
botforge/simplementation community (archive-listed) 2 samples, 1 ran (0 instrument-failed) · 1 unverified
Atari_DQN 2.DOUBLE-DQN_Deep-Reinforcement-Learning-with-Double-Q-Learning-van-Hasselt-etal-2015/double-dqn_atari.py ran · metamorphic tier: invariant no licence file found · pointer only · 6f4c156ef811e511 · report fixture: signature obs:Tensor/4/float · returned Tensor · level recorded 2026-09-22
_weights_init in 7 papers 2.DOUBLE-DQN_Deep-Reinforcement-Learning-with-Double-Q-Learning-van-Hasselt-etal-2015/double-dqn_atari.py unverified no licence file found · pointer only · 0645b4b6b8caa2a3 · report
daviddcho/supermario community (archive-listed) 5 samples, 4 ran (0 instrument-failed) · 1 unverified
DuelingDQN agent.py ran · metamorphic tier: deterministic no licence file found · pointer only · be74745caf4bf744 · report fixture: returned DuelingDQN · level recorded 2026-09-22
ExperienceReplay in 5 papers agent.py ran no licence file found · pointer only · 77c8ec6c865240fa · report fixture: returned ExperienceReplay · level recorded 2026-09-22
Logger in 25 papers agent.py ran no licence file found · pointer only · 908cf1edb8faa4b5 · report fixture: signature reward:float/0/float, distance:float/0/float, loss:float/0/float, q:float/0/float · returned NoneType · level recorded 2026-09-24
NoisyLinear in 9 papers agent.py ran · metamorphic tier: deterministic no licence file found · pointer only · 00c916fc3622888b · report fixture: returned NoisyLinear · level recorded 2026-09-22
DQNAgent agent.py unverified no licence file found · pointer only · 4c3c6bab1314cb4b · report
fengsterooni/dql community (archive-listed) 3 samples, 1 ran (0 instrument-failed) · 2 unverified
DeepQNetowrk DDQN/ddqn_agent.py ran · metamorphic tier: invariant no licence file found · pointer only · 85c944ddae8c9df8 · report fixture: signature state:Tensor/4/float · returned Tensor · level recorded 2026-09-22
DDQNAgent DDQN/ddqn_agent.py unverified no licence file found · pointer only · 49bf52df6b0d3ead · report
ReplayBuffer in 33 papers DDQN/ddqn_agent.py unverified no licence file found · pointer only · 15af333b2f369749 · report
guillaumeboniface/bananaland community (archive-listed) 4 samples, 3 ran (0 instrument-failed) · 1 unverified
AgentMemory agent.py ran no licence file found · pointer only · 570c12d7a6123152 · report fixture: returned AgentMemory · level recorded 2026-09-22
MLP in 375 papers agent.py ran no licence file found · pointer only · 5c959f3eff13eb01 · report fixture: returned MLP · level recorded 2026-09-22
Q_model agent.py ran · metamorphic tier: deterministic no licence file found · pointer only · 124226e5906f2192 · report fixture: returned Q_model · level recorded 2026-09-22
DLQAgent agent.py unverified no licence file found · pointer only · 8538006c32c0f609 · report
hamishs/JAX-RL community (archive-listed) 4 samples, 3 ran (0 instrument-failed) · 1 unverified
DQN in 8 papers src/jax_rl/algorithms/dqn.py ran MIT (permissive) · 0493df484647c5af · report fixture: returned DQN · level recorded 2026-09-22
ExperienceReplay in 5 papers src/jax_rl/algorithms/dqn.py ran MIT (permissive) · 986cc90d7e4eeaf2 · report fixture: returned ExperienceReplay · level recorded 2026-09-22
Transition in 7 papers src/jax_rl/algorithms/dqn.py ran MIT (permissive) · 1652d7453e55d633 · report fixture: returned Transition · level recorded 2026-09-22
DDQN src/jax_rl/algorithms/dqn.py unverified MIT (permissive) · 79adffedd37f2627 · report
hemilpanchiwala/Dueling-Network-Architectures community (archive-listed) 3 samples, 0 ran · 3 unverified
DuelingDDQNAgent DuelingDDQNAgent.py unverified no licence file found · pointer only · b028432950562bdb · report
DuelingDeepQNetwork DuelingDDQNAgent.py unverified no licence file found · pointer only · db8c50b16d68e727 · report
ExperienceReplayMemory DuelingDDQNAgent.py unverified no licence file found · pointer only · 1adc6959fb824443 · report
ifestus/rl community (archive-listed) 3 samples, 0 ran · 3 unverified
CNN in 23 papers dqn/dqn.py unverified no licence file found · pointer only · 39984e4ec10e928e · report
DDQN dqn/dqn.py unverified no licence file found · pointer only · dd41380cda3330bf · report
DQN in 8 papers dqn/dqn.py unverified no licence file found · pointer only · 2c00e8186282ff76 · report
jeffery1236/Atari_DoubleDeepQNetwork community (archive-listed) 3 samples, 0 ran · 3 unverified
DeepQNetwork DDQAgent.py unverified no licence file found · pointer only · 74d1f509eb3cf7cb · report
DoubleDQAgent DDQAgent.py unverified no licence file found · pointer only · b62844075843cfc9 · report
ReplayBuffer in 33 papers DDQAgent.py unverified no licence file found · pointer only · 28d5d41e56d53b63 · report
jezzarax/drlnd_p1_navigation community (archive-listed) 2 samples, 2 ran (0 instrument-failed) · 0 unverified
DDQNAgent agents.py ran no licence file found · pointer only · dc6c2d1d1cffa8c3 · report fixture: returned DDQNAgent · level recorded 2026-09-22
DummyAgent agents.py ran no licence file found · pointer only · 216682c7c24d1664 · report fixture: returned DummyAgent · level recorded 2026-09-22
jihoonerd/Deep-Reinforcement-Learning-with-Double-Q-learning community (archive-listed) 2 samples, 0 ran · 2 unverified
make_atari in 17 papers ddqn/utils/atari_wrappers.py unverified MIT (permissive) · a70a2154eb543432 · report
wrap_deepmind in 17 papers ddqn/utils/atari_wrappers.py unverified MIT (permissive) · 3b5c0e5667fd4eaa · report
kmdanielduan/DQN_Family_PyTorch community (archive-listed) 5 samples, 3 ran (0 instrument-failed) · 2 unverified
AgentConfig agent.py ran no licence file found · pointer only · ac9572223e61f5cf · report fixture: signature args:dict/0/? · returned NoneType · level recorded 2026-09-24
DQN in 8 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 249aaac189a4c179 · report fixture: returned DQN · level recorded 2026-09-22
ReplayMemory in 11 papers agent.py ran no licence file found · pointer only · 1a174f4ed5cef5f4 · report fixture: returned ReplayMemory · level recorded 2026-09-22
Agent in 42 papers agent.py unverified no licence file found · pointer only · 1324ea2444e296ba · report
EnvConfig agent.py unverified no licence file found · pointer only · cbdc121eed816162 · report
marload/DeepRL-TensorFlow2 community (archive-listed) 3 samples, 1 ran (0 instrument-failed) · 2 unverified
ReplayBuffer in 33 papers DoubleDQN/DoubleDQN_Discrete.py ran Apache-2.0 (permissive) · 56d9058560cd0b68 · report fixture: returned ReplayBuffer · level recorded 2026-09-22
ActionStateModel DoubleDQN/DoubleDQN_Discrete.py unverified Apache-2.0 (permissive) · 60f968e2c7d67881 · report
Agent in 42 papers DoubleDQN/DoubleDQN_Discrete.py unverified Apache-2.0 (permissive) · 684dac16bf6352e6 · report
matthewsparr/Deep-Zork community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
DQNAgent agent.py ran no licence file found · pointer only · c758b36e7d16e2ef · report fixture: returned DQNAgent · level recorded 2026-09-22
microsoft/med-deadend community (archive-listed) 5 samples, 3 ran (0 instrument-failed) · 2 unverified
QNetwork_128 rl.py ran · metamorphic tier: invariant fingerprinted MIT (permissive) · 308d7ccd559f5f3b · report fixture: signature x:Tensor/2/float · returned Tensor · level recorded 2026-09-22
QNetwork_64 rl.py ran · metamorphic tier: invariant fingerprinted MIT (permissive) · 24430ed056d60e5e · report fixture: signature x:Tensor/2/float · returned Tensor · level recorded 2026-09-22
QNetwork_6464 rl.py ran · metamorphic tier: invariant MIT (permissive) · 07905d0466e42900 · report fixture: signature x:Tensor/2/float · returned Tensor · level recorded 2026-09-22
RL rl.py unverified MIT (permissive) · 228f9f075525f9a2 · report
init_weights in 109 papers rl.py unverified MIT (permissive) · de8bc6ac984a0475 · report
mohit8935/Deep-Q-Learning-Paper community (archive-listed) 2 samples, 2 ran (0 instrument-failed) · 0 unverified
DDQNAgent DDQN/ddqn_agent.py ran no licence file found · pointer only · f145e7ca739738a0 · report fixture: returned DDQNAgent · level recorded 2026-09-22
ReplayBuffer in 33 papers DDQN/ddqn_agent.py ran no licence file found · pointer only · 41d448d208226341 · report fixture: returned ReplayBuffer · level recorded 2026-09-22
near32/regym community (archive-listed) 1 sample, 0 ran · 1 unverified
compute_loss in 97 papers
census: 11 implementations from 11 papers, 9 distinct outputs across 7 buckets, one shared input per bucket
regym/rl_algorithms/algorithms/DQN/ddqn_loss.py unverified MIT (permissive) · e04b6449372dffb4 · report
philtabor/Deep-Q-Learning-Paper-To-Code community (archive-listed) 3 samples, 3 ran (0 instrument-failed) · 0 unverified
DDQNAgent DDQN/ddqn_agent.py ran MIT (permissive) · 247f8b58095cdb6d · report fixture: returned DDQNAgent · level recorded 2026-09-22
DeepQNetwork DDQN/ddqn_agent.py ran · metamorphic tier: deterministic MIT (permissive) · 458d686bba2350d3 · report fixture: returned DeepQNetwork · level recorded 2026-09-22
ReplayBuffer in 33 papers DDQN/ddqn_agent.py ran MIT (permissive) · 2a2892343a3359ee · report fixture: returned ReplayBuffer · level recorded 2026-09-22
seacevedo/ReinforcementLearningProjects community (archive-listed) 2 samples, 2 ran (0 instrument-failed) · 0 unverified
Agent in 42 papers ddqn.py ran no licence file found · pointer only · 5f8524dc2a2cab41 · report fixture: returned Agent · level recorded 2026-09-22
ConvNet in 13 papers ddqn.py ran · metamorphic tier: deterministic no licence file found · pointer only · e8bea3ffc28d3e32 · report fixture: returned ConvNet · level recorded 2026-09-22
shehrum/RL_Navigation community (archive-listed) 2 samples, 0 ran · 2 unverified
Agent in 42 papers dqn_agent.py unverified no licence file found · pointer only · d258022cfdb2a560 · report
ReplayBuffer in 33 papers dqn_agent.py unverified no licence file found · pointer only · fd4b196396e41a0f · report
ssainz/reinforcement_learning_algorithms community (archive-listed) 3 samples, 2 ran (0 instrument-failed) · 1 unverified
ReplayMemory in 11 papers non_jupyter/Frozen_Lake_DDQN.py ran no licence file found · pointer only · 1d2063c27405ec4e · report fixture: returned ReplayMemory · level recorded 2026-09-22
q_net non_jupyter/Frozen_Lake_DDQN.py ran fingerprinted no licence file found · pointer only · c71291d513eadb0c · report fixture: returned q_net · level recorded 2026-09-22
optimize_model non_jupyter/Frozen_Lake_DDQN.py unverified no licence file found · pointer only · c596c533af422c46 · report
tensorlayer/RLzoo community (archive-listed) 2 samples, 0 ran · 2 unverified
DQN in 8 papers rlzoo/algorithms/dqn/dqn.py unverified Apache-2.0 (permissive) · 39cd1cea29282ec1 · report
PrioritizedReplayBuffer rlzoo/algorithms/dqn/dqn.py unverified Apache-2.0 (permissive) · 78e16e24720e0320 · report
tkcoding/DeepRL community (archive-listed) 1 sample, 0 ran · 1 unverified
Agent in 42 papers Banana_Navigator/Agent.py unverified no licence file found · pointer only · 59b97ac055efa363 · report
utarumo/RL_implementation community (archive-listed) 3 samples, 2 ran (0 instrument-failed) · 1 unverified
Memory in 19 papers dqn_atari.py ran MIT (permissive) · dc87a7e1418f9bb3 · report fixture: returned Memory · level recorded 2026-09-22
SumTree dqn_atari.py ran MIT (permissive) · f5e528a822875813 · report fixture: returned SumTree · level recorded 2026-09-22
Agent in 42 papers dqn_atari.py unverified MIT (permissive) · 1f87bb3ad2308635 · report
xgfelicia/Reinforcement-Learning community (archive-listed) 2 samples, 2 ran (0 instrument-failed) · 0 unverified
DQN in 8 papers Cartpole/double-dqn.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 8002739ef399b9f8 · report fixture: returned DQN · level recorded 2026-09-22
DQNagent Cartpole/double-dqn.py ran no licence file found · pointer only · bfb3e0ae3cba8f2e · report fixture: returned DQNagent · level recorded 2026-09-22
yukezhu/tensorflow-reinforce community (archive-listed) 1 sample, 0 ran · 1 unverified
NeuralQLearner rl/neural_q_learner.py unverified MIT (permissive) · 1124c41973a96953 · report
yzheng51/rl-dino-run community (archive-listed) 7 samples, 6 ran (0 instrument-failed) · 1 unverified
ConvNet in 13 papers agent.py ran · metamorphic tier: invariant MIT (permissive) · d7c5b75c3f5b0451 · report fixture: signature x:Tensor/4/float · returned Tensor · level recorded 2026-09-22
DQN in 8 papers agent.py ran MIT (permissive) · 4cd993bb1241d36c · report fixture: signature state:Tensor/4/float · returned Tensor · level recorded 2026-09-24
DoubleDQN agent.py ran fingerprinted MIT (permissive) · 99ac309143b7cf78 · report fixture: signature state:Tensor/4/float · returned Tensor · level recorded 2026-09-24
LinearAnneal agent.py ran MIT (permissive) · c852318db796f171 · report fixture: returned float · level recorded 2026-09-24
Net in 54 papers agent.py ran · metamorphic tier: invariant MIT (permissive) · 9ca97885778aa8a2 · report fixture: signature x:Tensor/4/float · returned Tensor · level recorded 2026-09-22
StateProcessor agent.py ran · metamorphic tier: invariant fingerprinted MIT (permissive) · 352c0c157af8514e · report fixture: signature state:ndarray/3/int · returned Tensor · level recorded 2026-09-22
HyperParam agent.py unverified MIT (permissive) · 5fcfb5dfeec4ae38 · report
OscarHuangWind/Preference-Guided-DQN-Atari community (archive-listed) 5 samples, 4 ran (0 instrument-failed) · 1 unverified
Net in 54 papers DRL.py ran · metamorphic tier: deterministic MIT (permissive) · 127094b1a1600120 · report fixture: returned Net · level recorded 2026-09-22
PER DRL.py ran MIT (permissive) · 0c36fb52695342c8 · report fixture: returned PER · level recorded 2026-09-22
ReplayMemory in 11 papers DRL.py ran MIT (permissive) · 61d763d7c26919c1 · report fixture: returned ReplayMemory · level recorded 2026-09-22
SumTree DRL.py ran MIT (permissive) · c4667f9f629308c9 · report fixture: returned SumTree · level recorded 2026-09-22
DQN in 8 papers DRL.py unverified MIT (permissive) · 5964c193c9302049 · report
deepmind/rlax community (archive-listed) 1 sample, 1 ran (1 instrument-failed) · 0 unverified
double_q_learning rlax/_src/value_learning.py ran; the contract check could not run (RAISES) fingerprinted Apache-2.0 (permissive) · df05e7e72c42e52a · report fixture: signature q_tm1:ArrayImpl/1/float, a_tm1:int/0/int, r_t:float/0/float, discount_t:float/0/float, q_t_value:ArrayImpl/1/float, q_t_selector:ArrayImpl/1/float, stop_target_gradients:bool/0/bool · returned ArrayImpl · level recorded 2026-09-22
hill-a/stable-baselines community (archive-listed) 7 samples, 0 ran · 7 unverified
absolute_scope_name stable_baselines/deepq/build_graph.py unverified MIT (permissive) · e407d6ccfa049980 · report
build_act in 5 papers stable_baselines/deepq/build_graph.py unverified MIT (permissive) · ca48ef0e750004ab · report
build_act_with_param_noise stable_baselines/deepq/build_graph.py unverified MIT (permissive) · 467a140347947f78 · report
build_train stable_baselines/deepq/build_graph.py unverified MIT (permissive) · 495764c8da29ad3c · report
default_param_noise_filter stable_baselines/deepq/build_graph.py unverified MIT (permissive) · f9790043f512484c · report
scope_name stable_baselines/deepq/build_graph.py unverified MIT (permissive) · e27ff7ad1ae26a0e · report
scope_vars stable_baselines/deepq/build_graph.py unverified MIT (permissive) · 7994dd5a6da7dcf7 · report

Method as code Syntology

What Syntology's graph records as this paper's claimed method, lifted into code: each assembly names its bricks (functions and classes), each with its own role and its own verification level. An assembly is not a check that the code implements the method correctly or reproduces the paper's results. Levels are each brick's own verification level in Syntology's graph (level 0: no recorded run), listed brick by brick and never combined into one level; “ran” means the brick executed on a synthesized input, not that its output is correct, and the instrument figure counts failures of Syntology's instrument, not of the code. Where a model read the code against the paper's claim, each assembly's label verdict is one model's judgement, kept in this page's JSON twin; it is not calibrated, so no card prints it. Siblings are assemblies lifted from this paper in the same batch for the same method: alternative readings kept side by side; none replaces or supersedes another.

36 assemblies on this paper: 36 harvested, 0 generated, listed in 1 groups (siblings lifted in one batch form one group).

36 sibling assemblies for Double Q-learning adaptation to DQN (harvested), lifted in the same batch on 2026-09-09: alternative readings kept side by side; none replaces or supersedes another. Bricks printed for 12 of 36 sibling assemblies; the other 24 are listed with their counts; every brick is in this page's JSON twin.

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 4 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 3 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. DLQAgent entry · level 0 · unverified · 8538006c32c0f609
  2. AgentMemory sampler · level 1 · ran · 570c12d7a6123152
  3. MLP model · level 1 · ran · 5c959f3eff13eb01
  4. Q_model model · level 1 · ran · 124226e5906f2192

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 2 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. DQNagent entry · level 1 · ran · bfb3e0ae3cba8f2e
  2. DQN model · level 1 · ran · 8002739ef399b9f8

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 5 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 3 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. Agent entry · level 0 · unverified · f370c556306eb621
  2. Qnetwork model · level 1 · ran · 286e69c369a502d4
  3. ReplayBuffer sampler · level 0 · unverified · f07ed06c9c4c4e7c
  4. FixedSize_BinarySearchTree helper · level 1 · ran · 42487fb842418ae5
  5. Node helper · level 1 · ran · 896578e9d4f08299

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 7 bricks; individually on a fixture: 5 ran (of which 4 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified. The assembly has not been run end-to-end.

one of 36 siblings

  1. Agent entry · level 0 · unverified · 1324ea2444e296ba
  2. AgentConfig helper · level 1 · ran · ac9572223e61f5cf
  3. DQN model · level 1 · ran · 249aaac189a4c179
  4. EnvConfig helper · level 0 · unverified · cbdc121eed816162
  5. PrioritizedReplayMemory helper · level 1 · ran · e9a7eb1cb21539cf
  6. ReplayMemory helper · level 1 · ran · 1a174f4ed5cef5f4
  7. SumTree helper · level 1 · ran · b414f4c639de69a4

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 5 bricks; individually on a fixture: 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 4 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. DQN entry · level 0 · unverified · 5964c193c9302049
  2. Net model · level 1 · ran · 127094b1a1600120
  3. PER helper · level 1 · ran · 0c36fb52695342c8
  4. ReplayMemory sampler · level 1 · ran · 61d763d7c26919c1
  5. SumTree helper · level 1 · ran · c4667f9f629308c9

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 3 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. DDQNAgent entry · level 1 · ran · 247f8b58095cdb6d
  2. DeepQNetwork model · level 1 · ran · 458d686bba2350d3
  3. ReplayBuffer helper · level 1 · ran · 2a2892343a3359ee

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 2 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. Agent entry · level 1 · ran · 5f8524dc2a2cab41
  2. ConvNet model · level 1 · ran · e8bea3ffc28d3e32

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 1 brick; individually on a fixture: 0 ran · 1 unverified. The assembly has not been run end-to-end.

one of 36 siblings

  1. compute_loss entry · level 0 · unverified · e04b6449372dffb4

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 4 bricks; individually on a fixture: 3 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified. The assembly has not been run end-to-end.

one of 36 siblings

  1. DDQNAgent entry · level 1 · ran · 00919eeae2ebc39b
  2. DQNAgent helper · level 0 · unverified · b9e4d950b7bc9105
  3. QNetwork model · level 1 · ran · f9462232ec862c47
  4. ReplayBuffer sampler · level 1 · ran · dcd7958b0ad16526

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 1 brick; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. DQNAgent entry · level 1 · ran · c758b36e7d16e2ef

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 5 bricks; individually on a fixture: 4 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified. The assembly has not been run end-to-end.

one of 36 siblings

  1. DQNAgent entry · level 0 · unverified · 4c3c6bab1314cb4b
  2. DuelingDQN model · level 1 · ran · be74745caf4bf744
  3. ExperienceReplay helper · level 1 · ran · 77c8ec6c865240fa
  4. Logger helper · level 1 · ran · 908cf1edb8faa4b5
  5. NoisyLinear model · level 1 · ran · 00c916fc3622888b

harvested Assembly for Double Q-learning adaptation to DQN (graph method: Double Q-learning adaptation to DQN). 2 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 36 siblings

  1. NeuralQLearner entry · level 0 · unverified · 1124c41973a96953
  2. ReplayBuffer helper · level 1 · ran · 3c7178cf937ba425
  1. DDQNAgent · 418a9fdedd6f143c · 2 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[12] in the JSON twin
  2. Agent · 4533c1faaa51c7c3 · 3 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; the one sample that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[13] in the JSON twin
  3. optimize_model · 8c9711219c733bea · 3 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 2 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[14] in the JSON twin
  4. DoubleDQN · 91839b7a9f7ac559 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[15] in the JSON twin
  5. DDQNAgent · 935a68e5782ff689 · 3 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[16] in the JSON twin
  6. Agent · 9d43f21ef62ea84e · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[17] in the JSON twin
  7. Agent · a1f9a00be7d56b83 · 3 bricks; individually on a fixture: 3 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[18] in the JSON twin
  8. Agent · b78bb8ab268a67df · 3 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 2 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[19] in the JSON twin
  9. DDQN · bac6b8699d5f40e5 · 6 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified; every one of the 3 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[20] in the JSON twin
  10. DDQNAgent · be88cef79e21ed75 · 3 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[21] in the JSON twin
  11. Atari_DQN · bfd281080d78b238 · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[22] in the JSON twin
  12. Agent · c0984caf393c969a · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[23] in the JSON twin
  13. Agent · c20bb9a43e23d964 · 3 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[24] in the JSON twin
  14. DoubleDQAgent · c7baf1411ba29942 · 3 bricks; individually on a fixture: 0 ran · 3 unverified · The assembly has not been run end-to-end · assemblies[25] in the JSON twin
  15. RL · d30153897f19e08a · 5 bricks; individually on a fixture: 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[26] in the JSON twin
  16. build_train · d48ca14ed25e1ef4 · 7 bricks; individually on a fixture: 0 ran · 7 unverified · The assembly has not been run end-to-end · assemblies[27] in the JSON twin
  17. double_q_learning · d6b48d3ca1137907 · 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[28] in the JSON twin
  18. DQNAgent · e06baffaddad26d7 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[29] in the JSON twin
  19. DuelingDDQNAgent · e0e70cecb703dc3c · 3 bricks; individually on a fixture: 0 ran · 3 unverified · The assembly has not been run end-to-end · assemblies[30] in the JSON twin
  20. DDQN · e3dd46df59de8bde · 3 bricks; individually on a fixture: 0 ran · 3 unverified · The assembly has not been run end-to-end · assemblies[31] in the JSON twin
  21. Agent · e6407346b86a3783 · 3 bricks; individually on a fixture: 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[32] in the JSON twin
  22. Agent · e870b1cc103a6dd5 · 3 bricks; individually on a fixture: 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[33] in the JSON twin
  23. DQN · f49b223a7fadc26b · 6 bricks; individually on a fixture: 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 4 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[34] in the JSON twin
  24. DoubleDQN · f9217da6f5614c4d · 8 bricks; individually on a fixture: 7 ran (of which 1 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[35] in the JSON twin

Each brick ends with its code_sha256, Syntology's identity for that exact code; click one to copy the get_code call that returns it. For agents, on Syntology's MCP service: assembly(arxiv_id="1509.06461") returns these assemblies with each brick's role and wiring (how to connect).

Tasks

Atari GamesDeep Reinforcement LearningQ-LearningReinforcement LearningReinforcement Learning (RL)

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Atari Games Atari 2600 Alien DQN noop Score 1620.0 #31 of 49 Archive leaderboard report
Atari Games Atari 2600 Alien DDQN (tuned) hs Score 1033.4 #38 of 49 Archive leaderboard report
Atari Games Atari 2600 Alien Prior+Duel hs Score 823.7 #42 of 49 Archive leaderboard report
Atari Games Atari 2600 Alien DQN hs Score 634.0 #45 of 49 Archive leaderboard report
Atari Games Atari 2600 Amidar DQN noop Score 978.0 #26 of 48 Archive leaderboard report
Atari Games Atari 2600 Amidar Prior+Duel hs Score 238.4 #34 of 48 Archive leaderboard report
Atari Games Atari 2600 Amidar DQN hs Score 178.4 #41 of 48 Archive leaderboard report
Atari Games Atari 2600 Amidar DDQN (tuned) hs Score 169.1 #44 of 48 Archive leaderboard report
Atari Games Atari 2600 Assault Prior+Duel hs Score 10950.6 #17 of 45 Archive leaderboard report
Atari Games Atari 2600 Assault DDQN (tuned) hs Score 6060.8 #25 of 45 Archive leaderboard report
Atari Games Atari 2600 Assault DQN noop Score 4280.4 #30 of 45 Archive leaderboard report
Atari Games Atari 2600 Assault DQN hs Score 3489.3 #34 of 45 Archive leaderboard report
Atari Games Atari 2600 Asterix Prior+Duel hs Score 364200.0 #11 of 49 Archive leaderboard report
Atari Games Atari 2600 Asterix DDQN (tuned) hs Score 16837.0 #34 of 49 Archive leaderboard report
Atari Games Atari 2600 Asterix DQN noop Score 4359.0 #40 of 49 Archive leaderboard report
Atari Games Atari 2600 Asterix DQN hs Score 3170.5 #43 of 49 Archive leaderboard report
Atari Games Atari 2600 Asteroids DQN hs Score 1458.7 #34 of 43 Archive leaderboard report
Atari Games Atari 2600 Asteroids DQN noop Score 1364.5 #35 of 43 Archive leaderboard report
Atari Games Atari 2600 Asteroids DDQN (tuned) hs Score 1193.2 #36 of 43 Archive leaderboard report
Atari Games Atari 2600 Asteroids Prior+Duel hs Score 1021.9 #39 of 43 Archive leaderboard report
Atari Games Atari 2600 Atlantis Prior+Duel hs Score 423252.0 #27 of 42 Archive leaderboard report
Atari Games Atari 2600 Atlantis DDQN (tuned) hs Score 319688.0 #33 of 42 Archive leaderboard report
Atari Games Atari 2600 Atlantis DQN hs Score 292491.0 #35 of 42 Archive leaderboard report
Atari Games Atari 2600 Atlantis DQN noop Score 279987.0 #36 of 42 Archive leaderboard report
Atari Games Atari 2600 Bank Heist Prior+Duel hs Score 1004.6 #25 of 45 Archive leaderboard report
Atari Games Atari 2600 Bank Heist DDQN (tuned) hs Score 886.0 #30 of 45 Archive leaderboard report
Atari Games Atari 2600 Bank Heist DQN noop Score 455.0 #35 of 45 Archive leaderboard report
Atari Games Atari 2600 Bank Heist DQN hs Score 312.7 #38 of 45 Archive leaderboard report
Atari Games Atari 2600 Battle Zone Prior+Duel hs Score 30650.0 #25 of 47 Archive leaderboard report
Atari Games Atari 2600 Battle Zone DQN noop Score 29900.0 #26 of 47 Archive leaderboard report
Atari Games Atari 2600 Battle Zone DDQN (tuned) hs Score 24740.0 #33 of 47 Archive leaderboard report
Atari Games Atari 2600 Battle Zone DQN hs Score 23750.0 #34 of 47 Archive leaderboard report
Atari Games Atari 2600 Beam Rider Prior+Duel hs Score 37412.2 #10 of 49 Archive leaderboard report
Atari Games Atari 2600 Beam Rider DDQN (tuned) hs Score 17417.2 #23 of 49 Archive leaderboard report
Atari Games Atari 2600 Beam Rider DQN hs Score 9743.2 #33 of 49 Archive leaderboard report
Atari Games Atari 2600 Beam Rider DQN noop Score 8627.5 #34 of 49 Archive leaderboard report
Atari Games Atari 2600 Berzerk Prior+Duel hs Score 2178.6 #16 of 39 Archive leaderboard report
Atari Games Atari 2600 Berzerk DDQN (tuned) hs Score 1011.1 #28 of 39 Archive leaderboard report
Atari Games Atari 2600 Berzerk DQN noop Score 585.6 #37 of 39 Archive leaderboard report
Atari Games Atari 2600 Berzerk DQN hs Score 493.4 #39 of 39 Archive leaderboard report
Atari Games Atari 2600 Bowling DDQN (tuned) hs Score 69.6 #19 of 44 Archive leaderboard report
Atari Games Atari 2600 Bowling DQN hs Score 56.5 #27 of 44 Archive leaderboard report
Atari Games Atari 2600 Bowling DQN noop Score 50.4 #30 of 44 Archive leaderboard report
Atari Games Atari 2600 Bowling Prior+Duel hs Score 50.4 #31 of 44 Archive leaderboard report
Atari Games Atari 2600 Boxing DQN noop Score 88.0 #30 of 45 Archive leaderboard report
Atari Games Atari 2600 Boxing Prior+Duel hs Score 79.2 #31 of 45 Archive leaderboard report
Atari Games Atari 2600 Boxing DDQN (tuned) hs Score 73.5 #34 of 45 Archive leaderboard report
Atari Games Atari 2600 Boxing DQN hs Score 70.3 #37 of 45 Archive leaderboard report
Atari Games Atari 2600 Breakout DQN noop Score 385.5 #28 of 58 Archive leaderboard report
Atari Games Atari 2600 Breakout DDQN (tuned) hs Score 368.9 #31 of 58 Archive leaderboard report
Atari Games Atari 2600 Breakout Prior+Duel hs Score 354.6 #34 of 58 Archive leaderboard report
Atari Games Atari 2600 Breakout DQN hs Score 354.5 #35 of 58 Archive leaderboard report
Atari Games Atari 2600 Centipede Prior+Duel hs Score 5570.2 #28 of 45 Archive leaderboard report
Atari Games Atari 2600 Centipede DQN noop Score 4657.7 #31 of 45 Archive leaderboard report
Atari Games Atari 2600 Centipede DQN hs Score 3973.9 #37 of 45 Archive leaderboard report
Atari Games Atari 2600 Centipede DDQN (tuned) hs Score 3853.5 #39 of 45 Archive leaderboard report
Atari Games Atari 2600 Chopper Command Prior+Duel hs Score 8058.0 #22 of 45 Archive leaderboard report
Atari Games Atari 2600 Chopper Command DQN noop Score 6126.0 #27 of 45 Archive leaderboard report
Atari Games Atari 2600 Chopper Command DQN hs Score 5017.0 #32 of 45 Archive leaderboard report
Atari Games Atari 2600 Chopper Command DDQN (tuned) hs Score 3495.0 #39 of 45 Archive leaderboard report
Atari Games Atari 2600 Crazy Climber Prior+Duel hs Score 127853.0 #26 of 49 Archive leaderboard report
Atari Games Atari 2600 Crazy Climber DDQN (tuned) hs Score 113782.0 #34 of 49 Archive leaderboard report
Atari Games Atari 2600 Crazy Climber DQN noop Score 110763.0 #36 of 49 Archive leaderboard report
Atari Games Atari 2600 Crazy Climber DQN hs Score 98128.0 #39 of 49 Archive leaderboard report
Atari Games Atari 2600 Demon Attack Prior+Duel hs Score 73371.3 #22 of 46 Archive leaderboard report
Atari Games Atari 2600 Demon Attack DDQN (tuned) hs Score 69803.4 #26 of 46 Archive leaderboard report
Atari Games Atari 2600 Demon Attack DQN hs Score 12550.7 #37 of 46 Archive leaderboard report
Atari Games Atari 2600 Demon Attack DQN noop Score 12149.4 #38 of 46 Archive leaderboard report
Atari Games Atari 2600 Double Dunk DQN hs Score -6.0 #29 of 43 Archive leaderboard report
Atari Games Atari 2600 Double Dunk DQN noop Score -6.6 #30 of 43 Archive leaderboard report
Atari Games Atari 2600 Double Dunk DDQN (tuned) hs Score -0.3 #33 of 43 Archive leaderboard report
Atari Games Atari 2600 Double Dunk Prior+Duel hs Score -10.7 #34 of 43 Archive leaderboard report
Atari Games Atari 2600 Enduro Prior+Duel hs Score 2223.9 #14 of 48 Archive leaderboard report
Atari Games Atari 2600 Enduro DDQN (tuned) hs Score 1216.6 #27 of 48 Archive leaderboard report
Atari Games Atari 2600 Enduro DQN noop Score 729.0 #32 of 48 Archive leaderboard report
Atari Games Atari 2600 Enduro DQN hs Score 626.7 #34 of 48 Archive leaderboard report
Atari Games Atari 2600 Fishing Derby Prior+Duel hs Score 17.0 #29 of 44 Archive leaderboard report
Atari Games Atari 2600 Fishing Derby DDQN (tuned) hs Score 3.2 #35 of 44 Archive leaderboard report
Atari Games Atari 2600 Fishing Derby DQN hs Score -1.6 #38 of 44 Archive leaderboard report
Atari Games Atari 2600 Fishing Derby DQN noop Score -4.9 #39 of 44 Archive leaderboard report
Atari Games Atari 2600 Freeway DQN noop Score 30.8 #31 of 59 Archive leaderboard report
Atari Games Atari 2600 Freeway DDQN (tuned) hs Score 28.8 #38 of 59 Archive leaderboard report
Atari Games Atari 2600 Freeway Prior+Duel hs Score 28.2 #39 of 59 Archive leaderboard report
Atari Games Atari 2600 Freeway DQN hs Score 26.9 #43 of 59 Archive leaderboard report
Atari Games Atari 2600 Frostbite Prior+Duel hs Score 4038.4 #20 of 53 Archive leaderboard report
Atari Games Atari 2600 Frostbite DDQN (tuned) hs Score 1448.1 #33 of 53 Archive leaderboard report
Atari Games Atari 2600 Frostbite DQN noop Score 797.4 #36 of 53 Archive leaderboard report
Atari Games Atari 2600 Frostbite DQN hs Score 496.1 #39 of 53 Archive leaderboard report
Atari Games Atari 2600 Gopher Prior+Duel hs Score 105148.4 #10 of 43 Archive leaderboard report
Atari Games Atari 2600 Gopher DDQN (tuned) hs Score 15253.0 #27 of 43 Archive leaderboard report
Atari Games Atari 2600 Gopher DQN noop Score 8777.4 #32 of 43 Archive leaderboard report
Atari Games Atari 2600 Gopher DQN hs Score 8190.4 #35 of 43 Archive leaderboard report
Atari Games Atari 2600 Gravitar DQN noop Score 473.0 #33 of 53 Archive leaderboard report
Atari Games Atari 2600 Gravitar DQN hs Score 298.0 #44 of 53 Archive leaderboard report
Atari Games Atari 2600 Gravitar DDQN (tuned) hs Score 200.5 #52 of 53 Archive leaderboard report
Atari Games Atari 2600 Gravitar Prior+Duel hs Score 167.0 #53 of 53 Archive leaderboard report
Atari Games Atari 2600 HERO DQN noop Score 20437.8 #29 of 44 Archive leaderboard report
Atari Games Atari 2600 HERO Prior+Duel hs Score 15459.2 #32 of 44 Archive leaderboard report
Atari Games Atari 2600 HERO DQN hs Score 14992.9 #34 of 44 Archive leaderboard report
Atari Games Atari 2600 HERO DDQN (tuned) hs Score 14892.5 #35 of 44 Archive leaderboard report
Atari Games Atari 2600 Ice Hockey Prior+Duel hs Score 0.5 #18 of 43 Archive leaderboard report
Atari Games Atari 2600 Ice Hockey DQN hs Score -1.6 #23 of 43 Archive leaderboard report
Atari Games Atari 2600 Ice Hockey DQN noop Score -1.9 #24 of 43 Archive leaderboard report
Atari Games Atari 2600 Ice Hockey DDQN (tuned) hs Score -2.5 #27 of 43 Archive leaderboard report
Atari Games Atari 2600 James Bond DQN noop Score 768.5 #28 of 45 Archive leaderboard report
Atari Games Atari 2600 James Bond DQN hs Score 697.5 #29 of 45 Archive leaderboard report
Atari Games Atari 2600 James Bond Prior+Duel hs Score 585.0 #32 of 45 Archive leaderboard report
Atari Games Atari 2600 James Bond DDQN (tuned) hs Score 573.0 #34 of 45 Archive leaderboard report
Atari Games Atari 2600 Kangaroo DDQN (tuned) hs Score 11204.0 #22 of 47 Archive leaderboard report
Atari Games Atari 2600 Kangaroo DQN noop Score 7259.0 #26 of 47 Archive leaderboard report
Atari Games Atari 2600 Kangaroo DQN hs Score 4496.0 #29 of 47 Archive leaderboard report
Atari Games Atari 2600 Kangaroo Prior+Duel hs Score 861.0 #41 of 47 Archive leaderboard report
Atari Games Atari 2600 Krull DQN noop Score 8422.3 #27 of 45 Archive leaderboard report
Atari Games Atari 2600 Krull Prior+Duel hs Score 7658.6 #33 of 45 Archive leaderboard report
Atari Games Atari 2600 Krull DDQN (tuned) hs Score 6796.1 #35 of 45 Archive leaderboard report
Atari Games Atari 2600 Krull DQN hs Score 6206.0 #37 of 45 Archive leaderboard report
Atari Games Atari 2600 Kung-Fu Master Prior+Duel hs Score 37484.0 #23 of 43 Archive leaderboard report
Atari Games Atari 2600 Kung-Fu Master DDQN (tuned) hs Score 30207.0 #32 of 43 Archive leaderboard report
Atari Games Atari 2600 Kung-Fu Master DQN noop Score 26059.0 #36 of 43 Archive leaderboard report
Atari Games Atari 2600 Kung-Fu Master DQN hs Score 20882.0 #39 of 43 Archive leaderboard report
Atari Games Atari 2600 Montezuma's Revenge DQN hs Score 47.0 #34 of 50 Archive leaderboard report
Atari Games Atari 2600 Montezuma's Revenge DDQN (tuned) hs Score 42.0 #35 of 50 Archive leaderboard report
Atari Games Atari 2600 Montezuma's Revenge Prior+Duel hs Score 24.0 #37 of 50 Archive leaderboard report
Atari Games Atari 2600 Ms. Pacman DQN noop Score 3085.6 #27 of 47 Archive leaderboard report
Atari Games Atari 2600 Ms. Pacman DDQN (tuned) hs Score 1241.3 #40 of 47 Archive leaderboard report
Atari Games Atari 2600 Ms. Pacman DQN hs Score 1092.3 #42 of 47 Archive leaderboard report
Atari Games Atari 2600 Ms. Pacman Prior+Duel hs Score 1007.8 #43 of 47 Archive leaderboard report
Atari Games Atari 2600 Name This Game Prior+Duel hs Score 13637.9 #19 of 43 Archive leaderboard report
Atari Games Atari 2600 Name This Game DDQN (tuned) hs Score 8960.3 #33 of 43 Archive leaderboard report
Atari Games Atari 2600 Name This Game DQN noop Score 8207.8 #34 of 43 Archive leaderboard report
Atari Games Atari 2600 Name This Game DQN hs Score 6738.8 #36 of 43 Archive leaderboard report
Atari Games Atari 2600 Pong DQN noop Score 19.5 #34 of 52 Archive leaderboard report
Atari Games Atari 2600 Pong DDQN (tuned) hs Score 19.1 #35 of 52 Archive leaderboard report
Atari Games Atari 2600 Pong Prior+Duel hs Score 18.4 #39 of 52 Archive leaderboard report
Atari Games Atari 2600 Pong DQN hs Score 18.0 #42 of 52 Archive leaderboard report
Atari Games Atari 2600 Private Eye Prior+Duel hs Score 1277.6 #22 of 52 Archive leaderboard report
Atari Games Atari 2600 Private Eye DQN hs Score 207.9 #32 of 52 Archive leaderboard report
Atari Games Atari 2600 Private Eye DQN noop Score 146.7 #39 of 52 Archive leaderboard report
Atari Games Atari 2600 Private Eye DDQN (tuned) hs Score -575.5 #52 of 52 Archive leaderboard report
Atari Games Atari 2600 Q*Bert Prior+Duel hs Score 14063.0 #35 of 57 Archive leaderboard report
Atari Games Atari 2600 Q*Bert DQN noop Score 13117.3 #37 of 57 Archive leaderboard report
Atari Games Atari 2600 Q*Bert DDQN (tuned) hs Score 11020.8 #38 of 57 Archive leaderboard report
Atari Games Atari 2600 Q*Bert DQN hs Score 9271.5 #41 of 57 Archive leaderboard report
Atari Games Atari 2600 River Raid Prior+Duel hs Score 16496.8 #18 of 41 Archive leaderboard report
Atari Games Atari 2600 River Raid DDQN (tuned) hs Score 10838.4 #27 of 41 Archive leaderboard report
Atari Games Atari 2600 River Raid DQN noop Score 7377.6 #32 of 41 Archive leaderboard report
Atari Games Atari 2600 River Raid DQN hs Score 4748.5 #36 of 41 Archive leaderboard report
Atari Games Atari 2600 Road Runner Prior+Duel hs Score 54630.0 #23 of 44 Archive leaderboard report
Atari Games Atari 2600 Road Runner DDQN (tuned) hs Score 43156.0 #30 of 44 Archive leaderboard report
Atari Games Atari 2600 Road Runner DQN noop Score 39544.0 #32 of 44 Archive leaderboard report
Atari Games Atari 2600 Road Runner DQN hs Score 35215.0 #34 of 44 Archive leaderboard report
Atari Games Atari 2600 Robotank DQN noop Score 63.9 #19 of 42 Archive leaderboard report
Atari Games Atari 2600 Robotank DDQN (tuned) hs Score 59.1 #25 of 42 Archive leaderboard report
Atari Games Atari 2600 Robotank DQN hs Score 58.7 #26 of 42 Archive leaderboard report
Atari Games Atari 2600 Robotank Prior+Duel hs Score 24.7 #34 of 42 Archive leaderboard report
Atari Games Atari 2600 Seaquest DDQN (tuned) hs Score 14498.0 #19 of 57 Archive leaderboard report
Atari Games Atari 2600 Seaquest DQN noop Score 5860.6 #30 of 57 Archive leaderboard report
Atari Games Atari 2600 Seaquest DQN hs Score 4216.7 #34 of 57 Archive leaderboard report
Atari Games Atari 2600 Seaquest Prior+Duel hs Score 1431.2 #44 of 57 Archive leaderboard report
Atari Games Atari 2600 Space Invaders Prior+Duel hs Score 8978.0 #18 of 55 Archive leaderboard report
Atari Games Atari 2600 Space Invaders DDQN (tuned) hs Score 2628.7 #32 of 55 Archive leaderboard report
Atari Games Atari 2600 Space Invaders DQN noop Score 1692.3 #39 of 55 Archive leaderboard report
Atari Games Atari 2600 Space Invaders DQN hs Score 1293.8 #41 of 55 Archive leaderboard report
Atari Games Atari 2600 Star Gunner Prior+Duel hs Score 127073.0 #14 of 43 Archive leaderboard report
Atari Games Atari 2600 Star Gunner DDQN (tuned) hs Score 58365.0 #28 of 43 Archive leaderboard report
Atari Games Atari 2600 Star Gunner DQN noop Score 54282.0 #31 of 43 Archive leaderboard report
Atari Games Atari 2600 Star Gunner DQN hs Score 52970.0 #32 of 43 Archive leaderboard report
Atari Games Atari 2600 Tennis DQN noop Score 12.2 #13 of 43 Archive leaderboard report
Atari Games Atari 2600 Tennis DQN hs Score 11.1 #15 of 43 Archive leaderboard report
Atari Games Atari 2600 Tennis DDQN (tuned) hs Score -7.8 #32 of 43 Archive leaderboard report
Atari Games Atari 2600 Tennis Prior+Duel hs Score -13.2 #33 of 43 Archive leaderboard report
Atari Games Atari 2600 Time Pilot DDQN (tuned) hs Score 6608.0 #31 of 44 Archive leaderboard report
Atari Games Atari 2600 Time Pilot Prior+Duel hs Score 4871.0 #37 of 44 Archive leaderboard report
Atari Games Atari 2600 Time Pilot DQN noop Score 4870.0 #38 of 44 Archive leaderboard report
Atari Games Atari 2600 Time Pilot DQN hs Score 4786.0 #40 of 44 Archive leaderboard report
Atari Games Atari 2600 Tutankham Prior+Duel hs Score 108.6 #36 of 44 Archive leaderboard report
Atari Games Atari 2600 Tutankham DDQN (tuned) hs Score 92.2 #38 of 44 Archive leaderboard report
Atari Games Atari 2600 Tutankham DQN noop Score 68.1 #39 of 44 Archive leaderboard report
Atari Games Atari 2600 Tutankham DQN hs Score 45.6 #42 of 44 Archive leaderboard report
Atari Games Atari 2600 Up and Down Prior+Duel hs Score 22681.3 #29 of 44 Archive leaderboard report
Atari Games Atari 2600 Up and Down DDQN (tuned) hs Score 19086.9 #31 of 44 Archive leaderboard report
Atari Games Atari 2600 Up and Down DQN noop Score 9989.9 #37 of 44 Archive leaderboard report
Atari Games Atari 2600 Up and Down DQN hs Score 8038.5 #40 of 44 Archive leaderboard report
Atari Games Atari 2600 Venture DQN noop Score 163.0 #30 of 55 Archive leaderboard report
Atari Games Atari 2600 Venture DQN hs Score 136.0 #31 of 55 Archive leaderboard report
Atari Games Atari 2600 Venture Prior+Duel hs Score 29.0 #41 of 55 Archive leaderboard report
Atari Games Atari 2600 Venture DDQN (tuned) hs Score 21.0 #44 of 55 Archive leaderboard report
Atari Games Atari 2600 Video Pinball Prior+Duel hs Score 447408.6 #20 of 42 Archive leaderboard report
Atari Games Atari 2600 Video Pinball DDQN (tuned) hs Score 367823.7 #21 of 42 Archive leaderboard report
Atari Games Atari 2600 Video Pinball DQN noop Score 196760.4 #27 of 42 Archive leaderboard report
Atari Games Atari 2600 Video Pinball DQN hs Score 154414.1 #29 of 42 Archive leaderboard report
Atari Games Atari 2600 Wizard of Wor Prior+Duel hs Score 10471.0 #18 of 41 Archive leaderboard report
Atari Games Atari 2600 Wizard of Wor DDQN (tuned) hs Score 6201.0 #29 of 41 Archive leaderboard report
Atari Games Atari 2600 Wizard of Wor DQN noop Score 2704.0 #37 of 41 Archive leaderboard report
Atari Games Atari 2600 Wizard of Wor DQN hs Score 1609.0 #39 of 41 Archive leaderboard report
Atari Games Atari 2600 Zaxxon Prior+Duel hs Score 11320.0 #23 of 41 Archive leaderboard report
Atari Games Atari 2600 Zaxxon DDQN (tuned) hs Score 8593.0 #32 of 41 Archive leaderboard report
Atari Games Atari 2600 Zaxxon DQN noop Score 5363.0 #35 of 41 Archive leaderboard report
Atari Games Atari 2600 Zaxxon DQN hs Score 4412.0 #37 of 41 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: DDQL

ConvolutionDDQLDQNDense ConnectionsDouble DQNDouble Q-learningExperience ReplayQ-Learning

2 archive method tags without a method page not shown.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections