{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adversarial-deep-reinforcement-learning-in","title":"Adversarial Deep Reinforcement Learning in Portfolio Management","arxiv_id":"1808.09940","date":"2018-08-29","proceeding":null,"authors":["Zhipeng Liang","Hao Chen","Junhao Zhu","Kangkang Jiang","Yan-ran Li"],"abstract":"In this paper, we implement three state-of-art continuous reinforcement\nlearning algorithms, Deep Deterministic Policy Gradient (DDPG), Proximal Policy\nOptimization (PPO) and Policy Gradient (PG)in portfolio management. All of them\nare widely-used in game playing and robot control. What's more, PPO has\nappealing theoretical propeties which is hopefully potential in portfolio\nmanagement. We present the performances of them under different settings,\nincluding different learning rates, objective functions, feature combinations,\nin order to provide insights for parameters tuning, features selection and data\npreparation. We also conduct intensive experiments in China Stock market and\nshow that PG is more desirable in financial market than DDPG and PPO, although\nboth of them are more advanced. What's more, we propose a so called Adversarial\nTraining method and show that it can greatly improve the training efficiency\nand significantly promote average daily return and sharpe ratio in back test.\nBased on this new modification, our experiments results show that our agent\nbased on Policy Gradient can outperform UCRP.","url_abs":"http://arxiv.org/abs/1808.09940v3","url_pdf":"http://arxiv.org/pdf/1808.09940v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adversarial-deep-reinforcement-learning-in","repo_url":"https://github.com/aleedelarica/XDRL-for-finance","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"adversarial-deep-reinforcement-learning-in","repo_url":"https://github.com/andreaslillevangbech/PortfolioManager-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"adversarial-deep-reinforcement-learning-in","repo_url":"https://github.com/bucky1995/Portfolio-RL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"adversarial-deep-reinforcement-learning-in","repo_url":"https://github.com/deepcrypto/Reinforcement-learning-in-portfolio-management-","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"adversarial-deep-reinforcement-learning-in","repo_url":"https://github.com/kftam1994/Robo_Advisor","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"management","task_name":"Management"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"ppo","method_name":"PPO"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.09940","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}