{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/jointly-pre-training-with-supervised","title":"Jointly Pre-training with Supervised, Autoencoder, and Value Losses for Deep Reinforcement Learning","arxiv_id":"1904.02206","date":"2019-04-03","proceeding":null,"authors":["Gabriel V. de la Cruz Jr.","Yunshu Du","Matthew E. Taylor"],"abstract":"Deep Reinforcement Learning (DRL) algorithms are known to be data\ninefficient. One reason is that a DRL agent learns both the feature and the\npolicy tabula rasa. Integrating prior knowledge into DRL algorithms is one way\nto improve learning efficiency since it helps to build helpful representations.\nIn this work, we consider incorporating human knowledge to accelerate the\nasynchronous advantage actor-critic (A3C) algorithm by pre-training a small\namount of non-expert human demonstrations. We leverage the supervised\nautoencoder framework and propose a novel pre-training strategy that jointly\ntrains a weighted supervised classification loss, an unsupervised\nreconstruction loss, and an expected return loss. The resulting pre-trained\nmodel learns more useful features compared to independently training in\nsupervised or unsupervised fashion. Our pre-training method drastically\nimproved the learning performance of the A3C agent in Atari games of Pong and\nMsPacman, exceeding the performance of the state-of-the-art algorithms at a\nmuch smaller number of game interactions. Our method is light-weight and easy\nto implement in a single machine. For reproducibility, our code is available at\ngithub.com/gabrieledcjr/DeepRL/tree/A3C-ALA2019","url_abs":"http://arxiv.org/abs/1904.02206v1","url_pdf":"http://arxiv.org/pdf/1904.02206v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"jointly-pre-training-with-supervised","repo_url":"https://github.com/gabrieledcjr/DeepRL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"a3c","method_name":"A3C"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}