{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/impala-scalable-distributed-deep-rl-with","title":"IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures","arxiv_id":"1802.01561","date":"2018-02-05","proceeding":"ICML 2018 7","authors":["Lasse Espeholt","Hubert Soyer","Remi Munos","Karen Simonyan","Volodymir Mnih","Tom Ward","Yotam Doron","Vlad Firoiu","Tim Harley","Iain Dunning","Shane Legg","Koray Kavukcuoglu"],"abstract":"In this work we aim to solve a large collection of tasks using a single\nreinforcement learning agent with a single set of parameters. A key challenge\nis to handle the increased amount of data and extended training time. We have\ndeveloped a new distributed agent IMPALA (Importance Weighted Actor-Learner\nArchitecture) that not only uses resources more efficiently in single-machine\ntraining but also scales to thousands of machines without sacrificing data\nefficiency or resource utilisation. We achieve stable learning at high\nthroughput by combining decoupled acting and learning with a novel off-policy\ncorrection method called V-trace. We demonstrate the effectiveness of IMPALA\nfor multi-task reinforcement learning on DMLab-30 (a set of 30 tasks from the\nDeepMind Lab environment (Beattie et al., 2016)) and Atari-57 (all available\nAtari games in Arcade Learning Environment (Bellemare et al., 2013a)). Our\nresults show that IMPALA is able to achieve better performance than previous\nagents with less data, and crucially exhibits positive transfer between tasks\nas a result of its multi-task approach.","url_abs":"http://arxiv.org/abs/1802.01561v3","url_pdf":"http://arxiv.org/pdf/1802.01561v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/deepmind/scalable_agent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/Sheepsody/Batched-Impala-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/crazydonkey200/neural-symbolic-machines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/deepmind/streetlearn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/facebookresearch/gala","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/facebookresearch/torchbeast","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/google-deepmind/scalable_agent","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/google-deepmind/streetlearn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/google-research/valan","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/haje01/impala","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/heiner/scalable_agent","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/jerrodparker20/adaptive-transformers-in-rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/michaelnny/deep_rl_zoo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/theSparta/neural-symbolic-machines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/threewisemonkeys-as/torched_impala","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/urw7rs/spiralpp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/villinvic/Georges","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/windstrip/DeepMind-StreetLearn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/deepmind/haiku","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"unanswered"}},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/deepmind/rlax","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/opendilab/DI-engine/blob/main/ding/policy/impala.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/ray-project/ray/tree/master/rllib","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/seolhokim/DistributedRL-Pytorch-Ray","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"impala-scalable-distributed-deep-rl-with","repo_url":"https://github.com/seolhokim/SimpleDistributedRL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"gradient-clipping","method_name":"Gradient Clipping"},{"method_slug":"impala","method_name":"IMPALA"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"v-trace","method_name":"V-trace"}],"datasets_introduced":[],"methods_introduced":[{"slug":"impala","name":"IMPALA","full_name":"IMPALA"},{"slug":"v-trace","name":"V-trace","full_name":"V-trace"}],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-alien","task":"Atari Games","dataset":"Atari 2600 Alien","model":"IMPALA (deep)","rank_in_archive_order":10,"of":49,"metrics":{"Score":"15962.10"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-amidar","task":"Atari Games","dataset":"Atari 2600 Amidar","model":"IMPALA (deep)","rank_in_archive_order":17,"of":48,"metrics":{"Score":"1554.79"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-assault","task":"Atari Games","dataset":"Atari 2600 Assault","model":"IMPALA (deep)","rank_in_archive_order":11,"of":45,"metrics":{"Score":"19148.47"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asterix","task":"Atari Games","dataset":"Atari 2600 Asterix","model":"IMPALA (deep)","rank_in_archive_order":16,"of":49,"metrics":{"Score":"300732.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asteroids","task":"Atari Games","dataset":"Atari 2600 Asteroids","model":"IMPALA (deep)","rank_in_archive_order":9,"of":43,"metrics":{"Score":"108590.05"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-atlantis","task":"Atari Games","dataset":"Atari 2600 Atlantis","model":"IMPALA (deep)","rank_in_archive_order":21,"of":42,"metrics":{"Score":"849967.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bank-heist","task":"Atari Games","dataset":"Atari 2600 Bank Heist","model":"IMPALA (deep)","rank_in_archive_order":16,"of":45,"metrics":{"Score":"1223.15"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-battle-zone","task":"Atari Games","dataset":"Atari 2600 Battle Zone","model":"IMPALA (deep)","rank_in_archive_order":36,"of":47,"metrics":{"Score":"20885.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-beam-rider","task":"Atari Games","dataset":"Atari 2600 Beam Rider","model":"IMPALA (deep)","rank_in_archive_order":12,"of":49,"metrics":{"Score":"32463.47"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-berzerk","task":"Atari Games","dataset":"Atari 2600 Berzerk","model":"IMPALA (deep)","rank_in_archive_order":18,"of":39,"metrics":{"Score":"1852.70"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bowling","task":"Atari Games","dataset":"Atari 2600 Bowling","model":"IMPALA (deep)","rank_in_archive_order":25,"of":44,"metrics":{"Score":"59.92"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-boxing","task":"Atari Games","dataset":"Atari 2600 Boxing","model":"IMPALA (deep)","rank_in_archive_order":10,"of":45,"metrics":{"Score":"99.96"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-breakout","task":"Atari Games","dataset":"Atari 2600 Breakout","model":"IMPALA (deep)","rank_in_archive_order":10,"of":58,"metrics":{"Score":"787.34"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-centipede","task":"Atari Games","dataset":"Atari 2600 Centipede","model":"IMPALA (deep)","rank_in_archive_order":18,"of":45,"metrics":{"Score":"11049.75"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-chopper-command","task":"Atari Games","dataset":"Atari 2600 Chopper Command","model":"IMPALA (deep)","rank_in_archive_order":12,"of":45,"metrics":{"Score":"28255.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-crazy-climber","task":"Atari Games","dataset":"Atari 2600 Crazy Climber","model":"IMPALA (deep)","rank_in_archive_order":22,"of":49,"metrics":{"Score":"136950.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-defender","task":"Atari Games","dataset":"Atari 2600 Defender","model":"IMPALA (deep)","rank_in_archive_order":11,"of":21,"metrics":{"Score":"185203.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-demon-attack","task":"Atari Games","dataset":"Atari 2600 Demon Attack","model":"IMPALA (deep)","rank_in_archive_order":10,"of":46,"metrics":{"Score":"132826.98"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-double-dunk","task":"Atari Games","dataset":"Atari 2600 Double Dunk","model":"IMPALA (deep)","rank_in_archive_order":38,"of":43,"metrics":{"Score":"-0.33"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-enduro","task":"Atari Games","dataset":"Atari 2600 Enduro","model":"IMPALA (deep)","rank_in_archive_order":45,"of":48,"metrics":{"Score":"0.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-fishing-derby","task":"Atari Games","dataset":"Atari 2600 Fishing Derby","model":"IMPALA (deep)","rank_in_archive_order":15,"of":44,"metrics":{"Score":"44.85"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-freeway","task":"Atari Games","dataset":"Atari 2600 Freeway","model":"IMPALA (deep)","rank_in_archive_order":59,"of":59,"metrics":{"Score":"0.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-frostbite","task":"Atari Games","dataset":"Atari 2600 Frostbite","model":"IMPALA (deep)","rank_in_archive_order":44,"of":53,"metrics":{"Score":"317.75"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gopher","task":"Atari Games","dataset":"Atari 2600 Gopher","model":"IMPALA (deep)","rank_in_archive_order":15,"of":43,"metrics":{"Score":"66782.30"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"IMPALA (deep)","rank_in_archive_order":40,"of":53,"metrics":{"Score":"359.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-hero","task":"Atari Games","dataset":"Atari 2600 HERO","model":"IMPALA (deep)","rank_in_archive_order":9,"of":44,"metrics":{"Score":"33730.55"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ice-hockey","task":"Atari Games","dataset":"Atari 2600 Ice Hockey","model":"IMPALA (deep)","rank_in_archive_order":14,"of":43,"metrics":{"Score":"3.48"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-james-bond","task":"Atari Games","dataset":"Atari 2600 James Bond","model":"IMPALA (deep)","rank_in_archive_order":31,"of":45,"metrics":{"Score":"601.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kangaroo","task":"Atari Games","dataset":"Atari 2600 Kangaroo","model":"IMPALA (deep)","rank_in_archive_order":35,"of":47,"metrics":{"Score":"1632.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-krull","task":"Atari Games","dataset":"Atari 2600 Krull","model":"IMPALA (deep)","rank_in_archive_order":28,"of":45,"metrics":{"Score":"8147.40"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kung-fu-master","task":"Atari Games","dataset":"Atari 2600 Kung-Fu Master","model":"IMPALA (deep)","rank_in_archive_order":19,"of":43,"metrics":{"Score":"43375.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"IMPALA (deep)","rank_in_archive_order":44,"of":50,"metrics":{"Score":"0.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ms-pacman","task":"Atari Games","dataset":"Atari 2600 Ms. Pacman","model":"IMPALA (deep)","rank_in_archive_order":12,"of":47,"metrics":{"Score":"7342.32"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-name-this-game","task":"Atari Games","dataset":"Atari 2600 Name This Game","model":"IMPALA (deep)","rank_in_archive_order":11,"of":43,"metrics":{"Score":"21537.20"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-phoenix","task":"Atari Games","dataset":"Atari 2600 Phoenix","model":"IMPALA (deep)","rank_in_archive_order":10,"of":21,"metrics":{"Score":"210996.45"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pitfall","task":"Atari Games","dataset":"Atari 2600 Pitfall!","model":"IMPALA (deep)","rank_in_archive_order":22,"of":23,"metrics":{"Score":"-1.66"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pong","task":"Atari Games","dataset":"Atari 2600 Pong","model":"IMPALA (deep)","rank_in_archive_order":15,"of":52,"metrics":{"Score":"20.98"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"IMPALA (deep)","rank_in_archive_order":48,"of":52,"metrics":{"Score":"98.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-qbert","task":"Atari Games","dataset":"Atari 2600 Q*Bert","model":"IMPALA (deep)","rank_in_archive_order":4,"of":57,"metrics":{"Score":"351200.12"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-river-raid","task":"Atari Games","dataset":"Atari 2600 River Raid","model":"IMPALA (deep)","rank_in_archive_order":6,"of":41,"metrics":{"Score":"29608.05"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-road-runner","task":"Atari Games","dataset":"Atari 2600 Road Runner","model":"IMPALA (deep)","rank_in_archive_order":19,"of":44,"metrics":{"Score":"57121.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-robotank","task":"Atari Games","dataset":"Atari 2600 Robotank","model":"IMPALA (deep)","rank_in_archive_order":36,"of":42,"metrics":{"Score":"12.96"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-seaquest","task":"Atari Games","dataset":"Atari 2600 Seaquest","model":"IMPALA (deep)","rank_in_archive_order":41,"of":57,"metrics":{"Score":"1753.20"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-skiing","task":"Atari Games","dataset":"Atari 2600 Skiing","model":"IMPALA (deep)","rank_in_archive_order":6,"of":23,"metrics":{"Score":"-10180.38"},"uses_additional_data":true},{"leaderboard":"/sota/atari-games-on-atari-2600-solaris","task":"Atari Games","dataset":"Atari 2600 Solaris","model":"IMPALA (deep)","rank_in_archive_order":19,"of":23,"metrics":{"Score":"2365.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-space-invaders","task":"Atari Games","dataset":"Atari 2600 Space Invaders","model":"IMPALA (deep)","rank_in_archive_order":9,"of":55,"metrics":{"Score":"43595.78"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-star-gunner","task":"Atari Games","dataset":"Atari 2600 Star Gunner","model":"IMPALA (deep)","rank_in_archive_order":8,"of":43,"metrics":{"Score":"200625.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-surround","task":"Atari Games","dataset":"Atari 2600 Surround","model":"IMPALA (deep)","rank_in_archive_order":8,"of":15,"metrics":{"Score":"7.56"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tennis","task":"Atari Games","dataset":"Atari 2600 Tennis","model":"IMPALA (deep)","rank_in_archive_order":19,"of":43,"metrics":{"Score":"0.55"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-time-pilot","task":"Atari Games","dataset":"Atari 2600 Time Pilot","model":"IMPALA (deep)","rank_in_archive_order":10,"of":44,"metrics":{"Score":"48481.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tutankham","task":"Atari Games","dataset":"Atari 2600 Tutankham","model":"IMPALA (deep)","rank_in_archive_order":11,"of":44,"metrics":{"Score":"292.11"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-up-and-down","task":"Atari Games","dataset":"Atari 2600 Up and Down","model":"IMPALA (deep)","rank_in_archive_order":11,"of":44,"metrics":{"Score":"332546.75"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"IMPALA (deep)","rank_in_archive_order":52,"of":55,"metrics":{"Score":"0.00"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-video-pinball","task":"Atari Games","dataset":"Atari 2600 Video Pinball","model":"IMPALA (deep)","rank_in_archive_order":13,"of":42,"metrics":{"Score":"572898.27"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-wizard-of-wor","task":"Atari Games","dataset":"Atari 2600 Wizard of Wor","model":"IMPALA (deep)","rank_in_archive_order":22,"of":41,"metrics":{"Score":"9157.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-yars-revenge","task":"Atari Games","dataset":"Atari 2600 Yars Revenge","model":"IMPALA (deep)","rank_in_archive_order":11,"of":17,"metrics":{"Score":"84231.14"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-zaxxon","task":"Atari Games","dataset":"Atari 2600 Zaxxon","model":"IMPALA (deep)","rank_in_archive_order":9,"of":41,"metrics":{"Score":"32935.50"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-games","task":"Atari Games","dataset":"Atari games","model":"IMPALA, deep","rank_in_archive_order":9,"of":12,"metrics":{"Mean Human Normalized Score":"957.34%"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-57","task":"Atari Games","dataset":"Atari-57","model":"IMPALA, deep","rank_in_archive_order":8,"of":11,"metrics":{"Human World Record Breakthrough":"3","Mean Human Normalized Score":"957.34%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.01561","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1802.01561"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/seolhokim/SimpleDistributedRL","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/rlax","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ray-project/ray/tree/master/rllib","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/deepmind/lab","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/streetlearn","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/crazydonkey200/neural-symbolic-machines","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/streetlearn","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/haiku","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jerrodparker20/adaptive-transformers-in-rl","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/torchbeast","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/scalable_agent","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/villinvic/Georges","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/windstrip/DeepMind-StreetLearn","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sheepsody/Batched-Impala-PyTorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/heiner/scalable_agent","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/valan","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/theSparta/neural-symbolic-machines","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/gala","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michaelnny/deep_rl_zoo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/seolhokim/DistributedRL-Pytorch-Ray","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/scalable_agent","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/haje01/impala","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/threewisemonkeys-as/torched_impala","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opendilab/DI-engine/blob/main/ding/policy/impala.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/urw7rs/spiralpp","reach":null}],"summary":{"ran":8,"ran_draft_wrong":2,"ran_fixture":4,"ran_honours":1,"ran_violates":1,"unverified":18},"by_repo_kind":{"listed":{"samples":34,"ran":16,"repositories":11}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"89ed626ce6f952a2","entry":"Categorical","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89ed626ce6f952a2"}},{"code_sha256_prefix":"e05eecb1a5750a6c","entry":"CategoricalActor","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e05eecb1a5750a6c"}},{"code_sha256_prefix":"4d9828f13202e523","entry":"ConfigLoader","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d9828f13202e523"}},{"code_sha256_prefix":"6490dcf1e1570c31","entry":"Distribution","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6490dcf1e1570c31"}},{"code_sha256_prefix":"aad00ea22af3c855","entry":"GALA_A2C","repo":"facebookresearch/gala","repo_kind":"listed","path":"gala/gala_a2c.py","file_url":"https://github.com/facebookresearch/gala/blob/HEAD/gala/gala_a2c.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"aad00ea22af3c855"}},{"code_sha256_prefix":"3fdd0c84f214c2bb","entry":"Learner","repo":"seolhokim/DistributedRL-Pytorch-Ray","repo_kind":"listed","path":"agents/runners/learners/impala_learner.py","file_url":"https://github.com/seolhokim/DistributedRL-Pytorch-Ray/blob/HEAD/agents/runners/learners/impala_learner.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fdd0c84f214c2bb"}},{"code_sha256_prefix":"7d75b7a3afdfddc6","entry":"V","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d75b7a3afdfddc6"}},{"code_sha256_prefix":"a7ffabde1fe6a3c9","entry":"VTrace","repo":"Sheepsody/Batched-Impala-PyTorch","repo_kind":"listed","path":"src/IMPALA.py","file_url":"https://github.com/Sheepsody/Batched-Impala-PyTorch/blob/HEAD/src/IMPALA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a7ffabde1fe6a3c9"}},{"code_sha256_prefix":"815b225c5ab9f314","entry":"action_log_probs","repo":"facebookresearch/torchbeast","repo_kind":"listed","path":"torchbeast/core/vtrace.py","file_url":"https://github.com/facebookresearch/torchbeast/blob/HEAD/torchbeast/core/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"815b225c5ab9f314"}},{"code_sha256_prefix":"123172de8a1bc477","entry":"from_importance_weights","repo":"facebookresearch/torchbeast","repo_kind":"listed","path":"torchbeast/core/vtrace.py","file_url":"https://github.com/facebookresearch/torchbeast/blob/HEAD/torchbeast/core/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"123172de8a1bc477"}},{"code_sha256_prefix":"715b439b8761ea76","entry":"from_logits","repo":"facebookresearch/torchbeast","repo_kind":"listed","path":"torchbeast/core/vtrace.py","file_url":"https://github.com/facebookresearch/torchbeast/blob/HEAD/torchbeast/core/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"715b439b8761ea76"}},{"code_sha256_prefix":"a872fa719c5242ff","entry":"log_probs_from_logits_and_actions","repo":"google-research/valan","repo_kind":"listed","path":"framework/vtrace.py","file_url":"https://github.com/google-research/valan/blob/HEAD/framework/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a872fa719c5242ff"}},{"code_sha256_prefix":"a75de4dc4c61d7dc","entry":"log_probs_from_logits_and_actions","repo":"google-deepmind/scalable_agent","repo_kind":"listed","path":"vtrace.py","file_url":"https://github.com/google-deepmind/scalable_agent/blob/HEAD/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a75de4dc4c61d7dc"}},{"code_sha256_prefix":"ba92bb44edd2ada6","entry":"pairwise_cross_corr","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba92bb44edd2ada6"}},{"code_sha256_prefix":"d9e1fb4c37c2e728","entry":"safe_crossover","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9e1fb4c37c2e728"}},{"code_sha256_prefix":"e565e1e08272948c","entry":"vtrace","repo":"deepmind/rlax","repo_kind":"listed","path":"rlax/_src/vtrace.py","file_url":"https://github.com/deepmind/rlax/blob/HEAD/rlax/_src/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e565e1e08272948c"}},{"code_sha256_prefix":"f34df61f30b311d5","entry":"AC","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f34df61f30b311d5"}},{"code_sha256_prefix":"103308756a178abe","entry":"ActionStateProbs","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"103308756a178abe"}},{"code_sha256_prefix":"ceddd134b3dd01f7","entry":"Default","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ceddd134b3dd01f7"}},{"code_sha256_prefix":"cd20ae279ac1aa9f","entry":"Impala","repo":"Sheepsody/Batched-Impala-PyTorch","repo_kind":"listed","path":"src/IMPALA.py","file_url":"https://github.com/Sheepsody/Batched-Impala-PyTorch/blob/HEAD/src/IMPALA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd20ae279ac1aa9f"}},{"code_sha256_prefix":"b3a1158468bd53aa","entry":"ImpalaLearner","repo":"seolhokim/DistributedRL-Pytorch-Ray","repo_kind":"listed","path":"agents/runners/learners/impala_learner.py","file_url":"https://github.com/seolhokim/DistributedRL-Pytorch-Ray/blob/HEAD/agents/runners/learners/impala_learner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3a1158468bd53aa"}},{"code_sha256_prefix":"cbb6758024239ec3","entry":"PlayerState","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbb6758024239ec3"}},{"code_sha256_prefix":"bc2ac55256ed974b","entry":"action_log_probs","repo":"urw7rs/spiralpp","repo_kind":"listed","path":"torchbeast/core/vtrace.py","file_url":"https://github.com/urw7rs/spiralpp/blob/HEAD/torchbeast/core/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bc2ac55256ed974b"}},{"code_sha256_prefix":"cb89e31851cc2931","entry":"calc_loss_and_backprop","repo":"haje01/impala","repo_kind":"listed","path":"learner.py","file_url":"https://github.com/haje01/impala/blob/HEAD/learner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb89e31851cc2931"}},{"code_sha256_prefix":"9f5db323eec0eeba","entry":"corr_matrix","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f5db323eec0eeba"}},{"code_sha256_prefix":"86f7ee994e64dc88","entry":"from_importance_weights","repo":"google-research/valan","repo_kind":"listed","path":"framework/vtrace.py","file_url":"https://github.com/google-research/valan/blob/HEAD/framework/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"86f7ee994e64dc88"}},{"code_sha256_prefix":"e11ee068514828e6","entry":"from_importance_weights","repo":"google-deepmind/scalable_agent","repo_kind":"listed","path":"vtrace.py","file_url":"https://github.com/google-deepmind/scalable_agent/blob/HEAD/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e11ee068514828e6"}},{"code_sha256_prefix":"c0ca8087e51d7e1d","entry":"from_logits","repo":"google-research/valan","repo_kind":"listed","path":"framework/vtrace.py","file_url":"https://github.com/google-research/valan/blob/HEAD/framework/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c0ca8087e51d7e1d"}},{"code_sha256_prefix":"e26d17c678348c08","entry":"from_logits","repo":"urw7rs/spiralpp","repo_kind":"listed","path":"torchbeast/core/vtrace.py","file_url":"https://github.com/urw7rs/spiralpp/blob/HEAD/torchbeast/core/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e26d17c678348c08"}},{"code_sha256_prefix":"2b5d90526b4ad312","entry":"from_logits","repo":"google-deepmind/scalable_agent","repo_kind":"listed","path":"vtrace.py","file_url":"https://github.com/google-deepmind/scalable_agent/blob/HEAD/vtrace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2b5d90526b4ad312"}},{"code_sha256_prefix":"b1527fc2ce13e346","entry":"get_vtrace_and_advantages","repo":"seolhokim/SimpleDistributedRL","repo_kind":"listed","path":"distributed_reinforcement_learning/algorithms/impala.py","file_url":"https://github.com/seolhokim/SimpleDistributedRL/blob/HEAD/distributed_reinforcement_learning/algorithms/impala.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1527fc2ce13e346"}},{"code_sha256_prefix":"a22618d583e622fe","entry":"layer_crossover","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a22618d583e622fe"}},{"code_sha256_prefix":"e1569fbaca09c124","entry":"nn_crossover","repo":"villinvic/Georges","repo_kind":"listed","path":"training/RL.py","file_url":"https://github.com/villinvic/Georges/blob/HEAD/training/RL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1569fbaca09c124"}},{"code_sha256_prefix":"01b62761de35ed90","entry":"train","repo":"seolhokim/SimpleDistributedRL","repo_kind":"listed","path":"distributed_reinforcement_learning/algorithms/impala.py","file_url":"https://github.com/seolhokim/SimpleDistributedRL/blob/HEAD/distributed_reinforcement_learning/algorithms/impala.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01b62761de35ed90"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}