{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/train-a-real-world-local-path-planner-in-one","title":"Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity","arxiv_id":"2305.04180","date":"2023-05-07","proceeding":null,"authors":["Jinghao Xin","Jinwoo Kim","Zhi Li","Ning li"],"abstract":"Deep Reinforcement Learning (DRL) has exhibited efficacy in resolving the Local Path Planning (LPP) problem. However, such application in the real world is immensely limited due to the deficient training efficiency and generalization capability of DRL. To alleviate these two issues, a solution named Color is proposed, which consists of an Actor-Sharer-Learner (ASL) training framework and a mobile robot-oriented simulator Sparrow. Specifically, the ASL intends to improve the training efficiency of DRL algorithms. It employs a Vectorized Data Collection (VDC) mode to expedite data acquisition, decouples the data collection from model optimization by multithreading, and partially connects the two procedures by harnessing a Time Feedback Mechanism (TFM) to evade data underuse or overuse. Meanwhile, the Sparrow simulator utilizes a 2D grid-based world, simplified kinematics, and conversion-free data flow to achieve a lightweight design. The lightness facilitates vectorized diversity, allowing diversified simulation setups across extensive copies of the vectorized environments, resulting in a notable enhancement in the generalization capability of the DRL algorithm being trained. Comprehensive experiments, comprising 57 DRL benchmark environments, 32 simulated and 36 real-world LPP scenarios, have been conducted to corroborate the superiority of our method in terms of efficiency and generalization. The code and the video of this paper are accessible at https://github.com/XinJingHao/Color.","url_abs":"https://arxiv.org/abs/2305.04180v3","url_pdf":"https://arxiv.org/pdf/2305.04180v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"train-a-real-world-local-path-planner-in-one","repo_url":"https://github.com/xinjinghao/color","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"robot-navigation","task_name":"Robot Navigation"}],"methods":[{"method_slug":"double-dqn","method_name":"Double DQN"},{"method_slug":"double-q-learning","method_name":"Double Q-learning"},{"method_slug":"experience-replay","method_name":"Experience Replay"}],"datasets_introduced":[{"slug":"sparrow","name":"Sparrow","full_name":"Sparrow-V0: A Reinforcement Learning Friendly Simulator for Mobile Robot"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-alien","task":"Atari Games","dataset":"Atari 2600 Alien","model":"ASL DDQN","rank_in_archive_order":14,"of":49,"metrics":{"Score":"6955.2"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-amidar","task":"Atari Games","dataset":"Atari 2600 Amidar","model":"ASL DDQN","rank_in_archive_order":11,"of":48,"metrics":{"Score":"2232.3"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-assault","task":"Atari Games","dataset":"Atari 2600 Assault","model":"ASL DDQN","rank_in_archive_order":14,"of":45,"metrics":{"Score":"14372.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asterix","task":"Atari Games","dataset":"Atari 2600 Asterix","model":"ASL DDQN","rank_in_archive_order":8,"of":49,"metrics":{"Score":"567640"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asteroids","task":"Atari Games","dataset":"Atari 2600 Asteroids","model":"ASL DDQN","rank_in_archive_order":27,"of":43,"metrics":{"Score":"1984.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-atlantis","task":"Atari Games","dataset":"Atari 2600 Atlantis","model":"ASL DDQN","rank_in_archive_order":16,"of":42,"metrics":{"Score":"947275"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bank-heist","task":"Atari Games","dataset":"Atari 2600 Bank Heist","model":"ASL DDQN","rank_in_archive_order":10,"of":45,"metrics":{"Score":"1340.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-battle-zone","task":"Atari Games","dataset":"Atari 2600 Battle Zone","model":"ASL DDQN","rank_in_archive_order":16,"of":47,"metrics":{"Score":"38986"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-beam-rider","task":"Atari Games","dataset":"Atari 2600 Beam Rider","model":"ASL DDQN","rank_in_archive_order":15,"of":49,"metrics":{"Score":"26841.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-berzerk","task":"Atari Games","dataset":"Atari 2600 Berzerk","model":"ASL DDQN","rank_in_archive_order":14,"of":39,"metrics":{"Score":"2597.2"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bowling","task":"Atari Games","dataset":"Atari 2600 Bowling","model":"ASL DDQN","rank_in_archive_order":23,"of":44,"metrics":{"Score":"62.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-boxing","task":"Atari Games","dataset":"Atari 2600 Boxing","model":"ASL DDQN","rank_in_archive_order":15,"of":45,"metrics":{"Score":"99.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-breakout","task":"Atari Games","dataset":"Atari 2600 Breakout","model":"ASL DDQN","rank_in_archive_order":18,"of":58,"metrics":{"Score":"621.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-centipede","task":"Atari Games","dataset":"Atari 2600 Centipede","model":"ASL DDQN","rank_in_archive_order":38,"of":45,"metrics":{"Score":"3899.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-chopper-command","task":"Atari Games","dataset":"Atari 2600 Chopper Command","model":"ASL DDQN","rank_in_archive_order":15,"of":45,"metrics":{"Score":"15071"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-crazy-climber","task":"Atari Games","dataset":"Atari 2600 Crazy Climber","model":"ASL DDQN","rank_in_archive_order":13,"of":49,"metrics":{"Score":"166019"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-defender","task":"Atari Games","dataset":"Atari 2600 Defender","model":"ASL DDQN","rank_in_archive_order":18,"of":21,"metrics":{"Score":"37026.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-demon-attack","task":"Atari Games","dataset":"Atari 2600 Demon Attack","model":"ASL DDQN","rank_in_archive_order":14,"of":46,"metrics":{"Score":"119773.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-double-dunk","task":"Atari Games","dataset":"Atari 2600 Double Dunk","model":"ASL DDQN","rank_in_archive_order":25,"of":43,"metrics":{"Score":"0.1"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-enduro","task":"Atari Games","dataset":"Atari 2600 Enduro","model":"ASL DDQN","rank_in_archive_order":16,"of":48,"metrics":{"Score":"2103.1"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-fishing-derby","task":"Atari Games","dataset":"Atari 2600 Fishing Derby","model":"ASL DDQN","rank_in_archive_order":21,"of":44,"metrics":{"Score":"35.1"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-freeway","task":"Atari Games","dataset":"Atari 2600 Freeway","model":"ASL DDQN","rank_in_archive_order":12,"of":59,"metrics":{"Score":"33.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-frostbite","task":"Atari Games","dataset":"Atari 2600 Frostbite","model":"ASL DDQN","rank_in_archive_order":12,"of":53,"metrics":{"Score":"8616.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gopher","task":"Atari Games","dataset":"Atari 2600 Gopher","model":"ASL DDQN","rank_in_archive_order":12,"of":43,"metrics":{"Score":"103514.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"ASL DDQN","rank_in_archive_order":26,"of":53,"metrics":{"Score":"760"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-hero","task":"Atari Games","dataset":"Atari 2600 HERO","model":"ASL DDQN","rank_in_archive_order":18,"of":44,"metrics":{"Score":"26578.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ice-hockey","task":"Atari Games","dataset":"Atari 2600 Ice Hockey","model":"ASL DDQN","rank_in_archive_order":43,"of":43,"metrics":{"Score":"-3.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-james-bond","task":"Atari Games","dataset":"Atari 2600 James Bond","model":"ASL DDQN","rank_in_archive_order":17,"of":45,"metrics":{"Score":"2237"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kangaroo","task":"Atari Games","dataset":"Atari 2600 Kangaroo","model":"ASL DDQN","rank_in_archive_order":17,"of":47,"metrics":{"Score":"13027"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-krull","task":"Atari Games","dataset":"Atari 2600 Krull","model":"ASL DDQN","rank_in_archive_order":17,"of":45,"metrics":{"Score":"10422.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kung-fu-master","task":"Atari Games","dataset":"Atari 2600 Kung-Fu Master","model":"ASL DDQN","rank_in_archive_order":11,"of":43,"metrics":{"Score":"85182"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"ASL DDQN","rank_in_archive_order":49,"of":50,"metrics":{"Score":"0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ms-pacman","task":"Atari Games","dataset":"Atari 2600 Ms. Pacman","model":"ASL DDQN","rank_in_archive_order":21,"of":47,"metrics":{"Score":"4416"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-name-this-game","task":"Atari Games","dataset":"Atari 2600 Name This Game","model":"ASL DDQN","rank_in_archive_order":13,"of":43,"metrics":{"Score":"16535.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-phoenix","task":"Atari Games","dataset":"Atari 2600 Phoenix","model":"ASL DDQN","rank_in_archive_order":12,"of":21,"metrics":{"Score":"71752.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pitfall","task":"Atari Games","dataset":"Atari 2600 Pitfall!","model":"ASL DDQN","rank_in_archive_order":19,"of":23,"metrics":{"Score":"0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pong","task":"Atari Games","dataset":"Atari 2600 Pong","model":"ASL DDQN","rank_in_archive_order":14,"of":52,"metrics":{"Score":"21"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"ASL DDQN","rank_in_archive_order":28,"of":52,"metrics":{"Score":"349.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-qbert","task":"Atari Games","dataset":"Atari 2600 Q*Bert","model":"ASL DDQN","rank_in_archive_order":17,"of":57,"metrics":{"Score":"24548.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-river-raid","task":"Atari Games","dataset":"Atari 2600 River Raid","model":"ASL DDQN","rank_in_archive_order":9,"of":41,"metrics":{"Score":"24445"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-road-runner","task":"Atari Games","dataset":"Atari 2600 Road Runner","model":"ASL DDQN","rank_in_archive_order":21,"of":44,"metrics":{"Score":"56520"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-robotank","task":"Atari Games","dataset":"Atari 2600 Robotank","model":"ASL DDQN","rank_in_archive_order":13,"of":42,"metrics":{"Score":"65.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-seaquest","task":"Atari Games","dataset":"Atari 2600 Seaquest","model":"ASL DDQN","rank_in_archive_order":14,"of":57,"metrics":{"Score":"29278.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-skiing","task":"Atari Games","dataset":"Atari 2600 Skiing","model":"ASL DDQN","rank_in_archive_order":23,"of":23,"metrics":{"Score":"-8295.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-solaris","task":"Atari Games","dataset":"Atari 2600 Solaris","model":"ASL DDQN","rank_in_archive_order":16,"of":23,"metrics":{"Score":"3506.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-space-invaders","task":"Atari Games","dataset":"Atari 2600 Space Invaders","model":"ASL DDQN","rank_in_archive_order":13,"of":55,"metrics":{"Score":"21602"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-star-gunner","task":"Atari Games","dataset":"Atari 2600 Star Gunner","model":"ASL DDQN","rank_in_archive_order":13,"of":43,"metrics":{"Score":"129140"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-surround","task":"Atari Games","dataset":"Atari 2600 Surround","model":"ASL DDQN","rank_in_archive_order":12,"of":15,"metrics":{"Score":"2.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tennis","task":"Atari Games","dataset":"Atari 2600 Tennis","model":"ASL DDQN","rank_in_archive_order":9,"of":43,"metrics":{"Score":"22.3"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-time-pilot","task":"Atari Games","dataset":"Atari 2600 Time Pilot","model":"ASL DDQN","rank_in_archive_order":19,"of":44,"metrics":{"Score":"12071"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tutankham","task":"Atari Games","dataset":"Atari 2600 Tutankham","model":"ASL DDQN","rank_in_archive_order":16,"of":44,"metrics":{"Score":"252.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-up-and-down","task":"Atari Games","dataset":"Atari 2600 Up and Down","model":"ASL DDQN","rank_in_archive_order":26,"of":44,"metrics":{"Score":"25127.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"ASL DDQN","rank_in_archive_order":26,"of":55,"metrics":{"Score":"291"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-video-pinball","task":"Atari Games","dataset":"Atari 2600 Video Pinball","model":"ASL DDQN","rank_in_archive_order":12,"of":42,"metrics":{"Score":"626794"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-wizard-of-wor","task":"Atari Games","dataset":"Atari 2600 Wizard of Wor","model":"ASL DDQN","rank_in_archive_order":12,"of":41,"metrics":{"Score":"21049"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-yars-revenge","task":"Atari Games","dataset":"Atari 2600 Yars Revenge","model":"ASL DDQN","rank_in_archive_order":13,"of":17,"metrics":{"Score":"29231.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-zaxxon","task":"Atari Games","dataset":"Atari 2600 Zaxxon","model":"ASL DDQN","rank_in_archive_order":15,"of":41,"metrics":{"Score":"16420"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}