{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vision-only-robot-navigation-in-a-neural","title":"Vision-Only Robot Navigation in a Neural Radiance World","arxiv_id":"2110.00168","date":"2021-10-01","proceeding":null,"authors":["Michal Adamkiewicz","Timothy Chen","Adam Caccavale","Rachel Gardner","Preston Culbertson","Jeannette Bohg","Mac Schwager"],"abstract":"Neural Radiance Fields (NeRFs) have recently emerged as a powerful paradigm for the representation of natural, complex 3D scenes. NeRFs represent continuous volumetric density and RGB values in a neural network, and generate photo-realistic images from unseen camera viewpoints through ray tracing. We propose an algorithm for navigating a robot through a 3D environment represented as a NeRF using only an on-board RGB camera for localization. We assume the NeRF for the scene has been pre-trained offline, and the robot's objective is to navigate through unoccupied space in the NeRF to reach a goal pose. We introduce a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF based on a discrete time version of differential flatness that is amenable to constraining the robot's full pose and control inputs. We also introduce an optimization based filtering method to estimate 6DoF pose and velocities for the robot in the NeRF given only an onboard RGB camera. We combine the trajectory planner with the pose filter in an online replanning loop to give a vision-based robot navigation pipeline. We present simulation results with a quadrotor robot navigating through a jungle gym environment, the inside of a church, and Stonehenge using only an RGB camera. We also demonstrate an omnidirectional ground robot navigating through the church, requiring it to reorient to fit through the narrow gap. Videos of this work can be found at https://mikh3x4.github.io/nerf-navigation/ .","url_abs":"https://arxiv.org/abs/2110.00168v2","url_pdf":"https://arxiv.org/pdf/2110.00168v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"vision-only-robot-navigation-in-a-neural","repo_url":"https://github.com/mikh3x4/nerf-navigation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2110.00168","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.00168"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mikh3x4/nerf-navigation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"listed":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"00979fadb02bf8df","entry":"huber_loss","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"loss.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00979fadb02bf8df"}},{"code_sha256_prefix":"df2164d2c1f74c1b","entry":"linear_to_srgb","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/utils.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df2164d2c1f74c1b"}},{"code_sha256_prefix":"ddcd1a350d11a260","entry":"mape_loss","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"loss.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ddcd1a350d11a260"}},{"code_sha256_prefix":"cc5a245ddce1b228","entry":"nerf_matrix_to_ngp","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/provider.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/provider.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc5a245ddce1b228"}},{"code_sha256_prefix":"fcb5606f5d82bb61","entry":"rand_poses","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/provider.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/provider.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fcb5606f5d82bb61"}},{"code_sha256_prefix":"8ed7b9fc7f33d1dd","entry":"sample_pdf","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/renderer.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/renderer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ed7b9fc7f33d1dd"}},{"code_sha256_prefix":"78534c794e6d4ae8","entry":"srgb_to_linear","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/utils.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78534c794e6d4ae8"}},{"code_sha256_prefix":"524cb67e717d865a","entry":"get_encoder","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"encoding.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/encoding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"524cb67e717d865a"}},{"code_sha256_prefix":"caae47939a5d20c0","entry":"get_rays","repo":"mikh3x4/nerf-navigation","repo_kind":"listed","path":"nerf/utils.py","file_url":"https://github.com/mikh3x4/nerf-navigation/blob/HEAD/nerf/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"caae47939a5d20c0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}