{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/active-perception-using-neural-radiance","title":"Active Perception using Neural Radiance Fields","arxiv_id":"2310.09892","date":"2023-10-15","proceeding":null,"authors":["Siming He","Christopher D. Hsu","Dexter Ong","Yifei Simon Shao","Pratik Chaudhari"],"abstract":"We study active perception from first principles to argue that an autonomous agent performing active perception should maximize the mutual information that past observations posses about future ones. Doing so requires (a) a representation of the scene that summarizes past observations and the ability to update this representation to incorporate new observations (state estimation and mapping), (b) the ability to synthesize new observations of the scene (a generative model), and (c) the ability to select control trajectories that maximize predictive information (planning). This motivates a neural radiance field (NeRF)-like representation which captures photometric, geometric and semantic properties of the scene grounded. This representation is well-suited to synthesizing new observations from different viewpoints. And thereby, a sampling-based planner can be used to calculate the predictive information from synthetic observations along dynamically-feasible trajectories. We use active perception for exploring cluttered indoor environments and employ a notion of semantic uncertainty to check for the successful completion of an exploration task. We demonstrate these ideas via simulation in realistic 3D indoor environments.","url_abs":"https://arxiv.org/abs/2310.09892v2","url_pdf":"https://arxiv.org/pdf/2310.09892v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"active-perception-using-neural-radiance","repo_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.09892","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.09892"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields","reach":{"status":"ok"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"0b52b3d42f6b1f78","entry":"atan_zero_to_twopi","repo":"grasp-lyrl/active-perception-using-neural-radiance-fields","repo_kind":"official","path":"perception/data_proc/depth_to_grid.py","file_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields/blob/HEAD/perception/data_proc/depth_to_grid.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b52b3d42f6b1f78"}},{"code_sha256_prefix":"79b68e2c928db301","entry":"bresenham","repo":"grasp-lyrl/active-perception-using-neural-radiance-fields","repo_kind":"official","path":"perception/data_proc/depth_to_grid.py","file_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields/blob/HEAD/perception/data_proc/depth_to_grid.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79b68e2c928db301"}},{"code_sha256_prefix":"1c283f140e9f5807","entry":"file_read","repo":"grasp-lyrl/active-perception-using-neural-radiance-fields","repo_kind":"official","path":"perception/data_proc/depth_to_grid.py","file_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields/blob/HEAD/perception/data_proc/depth_to_grid.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c283f140e9f5807"}},{"code_sha256_prefix":"4b11516c2d3435a5","entry":"similarity_from_cameras","repo":"grasp-lyrl/active-perception-using-neural-radiance-fields","repo_kind":"official","path":"perception/models/datasets/nerf_360_v2.py","file_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields/blob/HEAD/perception/models/datasets/nerf_360_v2.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b11516c2d3435a5"}},{"code_sha256_prefix":"9ca2b129b9c65828","entry":"contract_to_unisphere","repo":"grasp-lyrl/active-perception-using-neural-radiance-fields","repo_kind":"official","path":"perception/models/radiance_fields/ngp.py","file_url":"https://github.com/grasp-lyrl/active-perception-using-neural-radiance-fields/blob/HEAD/perception/models/radiance_fields/ngp.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ca2b129b9c65828"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}