{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-reinforcement-learning-1","title":"Benchmarking Reinforcement Learning Techniques for Autonomous Navigation","arxiv_id":"2210.04839","date":"2022-10-10","proceeding":null,"authors":["Zifan Xu","Bo Liu","Xuesu Xiao","Anirudh Nair","Peter Stone"],"abstract":"Deep reinforcement learning (RL) has brought many successes for autonomous robot navigation. However, there still exists important limitations that prevent real-world use of RL-based navigation systems. For example, most learning approaches lack safety guarantees; and learned navigation systems may not generalize well to unseen environments. Despite a variety of recent learning techniques to tackle these challenges in general, a lack of an open-source benchmark and reproducible learning methods specifically for autonomous navigation makes it difficult for roboticists to choose what learning methods to use for their mobile robots and for learning researchers to identify current shortcomings of general learning methods for autonomous navigation. In this paper, we identify four major desiderata of applying deep RL approaches for autonomous navigation: (D1) reasoning under uncertainty, (D2) safety, (D3) learning from limited trial-and-error data, and (D4) generalization to diverse and novel environments. Then, we explore four major classes of learning techniques with the purpose of achieving one or more of the four desiderata: memory-based neural network architectures (D1), safe RL (D2), model-based RL (D2, D3), and domain randomization (D4). By deploying these learning techniques in a new open-source large-scale navigation benchmark and real-world environments, we perform a comprehensive study aimed at establishing to what extent can these techniques achieve these desiderata for RL-based navigation systems.","url_abs":"https://arxiv.org/abs/2210.04839v2","url_pdf":"https://arxiv.org/pdf/2210.04839v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-reinforcement-learning-1","repo_url":"https://github.com/Daffan/ros_jackal","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-navigation","task_name":"Autonomous Navigation"},{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"robot-navigation","task_name":"Robot Navigation"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.04839","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.04839"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Daffan/ros_jackal","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7896c03dcc426f24","entry":"get_world_name","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"rl_algos/tester.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/rl_algos/tester.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7896c03dcc426f24"}},{"code_sha256_prefix":"8883a16afb4b4d46","entry":"load_policy","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"actor.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/actor.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8883a16afb4b4d46"}},{"code_sha256_prefix":"c51ea62545640750","entry":"load_policy","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"rl_algos/tester.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/rl_algos/tester.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c51ea62545640750"}},{"code_sha256_prefix":"00a33466c69c5705","entry":"load_tf_weights_in_gpt2","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"rl_algos/trajectory_gpt2.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/rl_algos/trajectory_gpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00a33466c69c5705"}},{"code_sha256_prefix":"c6c5be8f3a6820d0","entry":"run_actor_in_container","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"rl_algos/collector.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/rl_algos/collector.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6c5be8f3a6820d0"}},{"code_sha256_prefix":"df92ddf3513dd833","entry":"write_buffer","repo":"Daffan/ros_jackal","repo_kind":"listed","path":"actor.py","file_url":"https://github.com/Daffan/ros_jackal/blob/HEAD/actor.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df92ddf3513dd833"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}