{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decentralized-distributed-ppo-solving","title":"DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames","arxiv_id":"1911.00357","date":"2019-11-01","proceeding":"ICLR 2020 1","authors":["Erik Wijmans","Abhishek Kadian","Ari Morcos","Stefan Lee","Irfan Essa","Devi Parikh","Manolis Savva","Dhruv Batra"],"abstract":"We present Decentralized Distributed Proximal Policy Optimization (DD-PPO), a method for distributed reinforcement learning in resource-intensive simulated environments. DD-PPO is distributed (uses multiple machines), decentralized (lacks a centralized server), and synchronous (no computation is ever stale), making it conceptually simple and easy to implement. In our experiments on training virtual robots to navigate in Habitat-Sim, DD-PPO exhibits near-linear scaling -- achieving a speedup of 107x on 128 GPUs over a serial implementation. We leverage this scaling to train an agent for 2.5 Billion steps of experience (the equivalent of 80 years of human experience) -- over 6 months of GPU-time training in under 3 days of wall-clock time with 64 GPUs. This massive-scale training not only sets the state of art on Habitat Autonomous Navigation Challenge 2019, but essentially solves the task --near-perfect autonomous navigation in an unseen environment without access to a map, directly from an RGB-D camera and a GPS+Compass sensor. Fortuitously, error vs computation exhibits a power-law-like distribution; thus, 90% of peak performance is obtained relatively early (at 100 million steps) and relatively cheaply (under 1 day with 8 GPUs). Finally, we show that the scene understanding and navigation policies learned can be transferred to other navigation tasks -- the analog of ImageNet pre-training + task-specific fine-tuning for embodied AI. Our model outperforms ImageNet pre-trained CNNs on these transfer tasks and can serve as a universal resource (all models and code are publicly available).","url_abs":"https://arxiv.org/abs/1911.00357v2","url_pdf":"https://arxiv.org/pdf/1911.00357v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/facebookresearch/habitat-api","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/GT-RIPL/robo-vln","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/allenai/robothor-challenge","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/facebookresearch/habitat-api/tree/master/habitat_baselines/rl/ddppo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/jacobkrantz/VLN-CE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/yangysc/resinet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/opendilab/DI-engine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"decentralized-distributed-ppo-solving","repo_url":"https://github.com/ray-project/ray/tree/master/rllib","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"autonomous-navigation","task_name":"Autonomous Navigation"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"pointgoal-navigation","task_name":"PointGoal Navigation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"robot-navigation","task_name":"Robot Navigation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"}],"methods":[{"method_slug":"dd-ppo","method_name":"DD-PPO"}],"datasets_introduced":[],"methods_introduced":[{"slug":"dd-ppo","name":"DD-PPO","full_name":"Decentralized Distributed Proximal Policy Optimization"}],"results":[{"leaderboard":"/sota/pointgoal-navigation-on-gibson-pointgoal","task":"PointGoal Navigation","dataset":"Gibson PointGoal Navigation","model":"Depth DDPPO","rank_in_archive_order":1,"of":2,"metrics":{"spl":"0.917"},"uses_additional_data":false},{"leaderboard":"/sota/robot-navigation-on-habitat-2020-object-nav-1","task":"Robot Navigation","dataset":"Habitat 2020 Object Nav test-std","model":"RGBD+DD-PPO","rank_in_archive_order":6,"of":13,"metrics":{"DISTANCE_TO_GOAL":"9.31617","SOFT_SPL":"0.14718","SPL":"0.02119","SUCCESS":"0.06165"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1911.00357","atlas_url":"https://app.syntology.ai/?focus=1911.00357","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}