{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/playing-atari-with-six-neurons","title":"Playing Atari with Six Neurons","arxiv_id":"1806.01363","date":"2018-06-04","proceeding":null,"authors":["Giuseppe Cuccu","Julian Togelius","Philippe Cudre-Mauroux"],"abstract":"Deep reinforcement learning, applied to vision-based problems like Atari\ngames, maps pixels directly to actions; internally, the deep neural network\nbears the responsibility of both extracting useful information and making\ndecisions based on it. By separating the image processing from decision-making,\none could better understand the complexity of each task, as well as potentially\nfind smaller policy representations that are easier for humans to understand\nand may generalize better. To this end, we propose a new method for learning\npolicies and compact state representations separately but simultaneously for\npolicy approximation in reinforcement learning. State representations are\ngenerated by an encoder based on two novel algorithms: Increasing Dictionary\nVector Quantization makes the encoder capable of growing its dictionary size\nover time, to address new observations as they appear in an open-ended\nonline-learning context; Direct Residuals Sparse Coding encodes observations by\ndisregarding reconstruction error minimization, and aiming instead for highest\ninformation inclusion. The encoder autonomously selects observations online to\ntrain on, in order to maximize code sparsity. As the dictionary size increases,\nthe encoder produces increasingly larger inputs for the neural network: this is\naddressed by a variation of the Exponential Natural Evolution Strategies\nalgorithm which adapts its probability distribution dimensionality along the\nrun. We test our system on a selection of Atari games using tiny neural\nnetworks of only 6 to 18 neurons (depending on the game's controls). These are\nstill capable of achieving results comparable---and occasionally superior---to\nstate-of-the-art techniques which use two orders of magnitude more neurons.","url_abs":"http://arxiv.org/abs/1806.01363v2","url_pdf":"http://arxiv.org/pdf/1806.01363v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"playing-atari-with-six-neurons","repo_url":"https://github.com/giuse/DNE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-demon-attack","task":"Atari Games","dataset":"Atari 2600 Demon Attack","model":"IDVQ + DRSC + XNES","rank_in_archive_order":45,"of":46,"metrics":{"Score":"325"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-fishing-derby","task":"Atari Games","dataset":"Atari 2600 Fishing Derby","model":"IDVQ + DRSC + XNES","rank_in_archive_order":42,"of":44,"metrics":{"Score":"-10"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-frostbite","task":"Atari Games","dataset":"Atari 2600 Frostbite","model":"IDVQ + DRSC + XNES","rank_in_archive_order":46,"of":53,"metrics":{"Score":"300"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kangaroo","task":"Atari Games","dataset":"Atari 2600 Kangaroo","model":"IDVQ + DRSC + XNES","rank_in_archive_order":40,"of":47,"metrics":{"Score":"1200"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-name-this-game","task":"Atari Games","dataset":"Atari 2600 Name This Game","model":"IDVQ + DRSC + XNES","rank_in_archive_order":43,"of":43,"metrics":{"Score":"920"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-phoenix","task":"Atari Games","dataset":"Atari 2600 Phoenix","model":"IDVQ + DRSC + XNES","rank_in_archive_order":21,"of":21,"metrics":{"Score":"4600"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-qbert","task":"Atari Games","dataset":"Atari 2600 Q*Bert","model":"IDVQ + DRSC + XNES","rank_in_archive_order":48,"of":57,"metrics":{"Score":"1250"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-seaquest","task":"Atari Games","dataset":"Atari 2600 Seaquest","model":"IDVQ + DRSC + XNES","rank_in_archive_order":54,"of":57,"metrics":{"Score":"320"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-space-invaders","task":"Atari Games","dataset":"Atari 2600 Space Invaders","model":"IDVQ + DRSC + XNES","rank_in_archive_order":47,"of":55,"metrics":{"Score":"830"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-time-pilot","task":"Atari Games","dataset":"Atari 2600 Time Pilot","model":"IDVQ + DRSC + XNES","rank_in_archive_order":41,"of":44,"metrics":{"Score":"4600"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.01363","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}