{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cognitive-mapping-and-planning-for-visual","title":"Cognitive Mapping and Planning for Visual Navigation","arxiv_id":"1702.03920","date":"2017-02-13","proceeding":"CVPR 2017 7","authors":["Saurabh Gupta","Varun Tolani","James Davidson","Sergey Levine","Rahul Sukthankar","Jitendra Malik"],"abstract":"We introduce a neural architecture for navigation in novel environments. Our\nproposed architecture learns to map from first-person views and plans a\nsequence of actions towards goals in the environment. The Cognitive Mapper and\nPlanner (CMP) is based on two key ideas: a) a unified joint architecture for\nmapping and planning, such that the mapping is driven by the needs of the task,\nand b) a spatial memory with the ability to plan given an incomplete set of\nobservations about the world. CMP constructs a top-down belief map of the world\nand applies a differentiable neural net planner to produce the next action at\neach time step. The accumulated belief of the world enables the agent to track\nvisited regions of the environment. We train and test CMP on navigation\nproblems in simulation environments derived from scans of real world buildings.\nOur experiments demonstrate that CMP outperforms alternate learning-based\narchitectures, as well as, classical mapping and path planning approaches in\nmany cases. Furthermore, it naturally extends to semantically specified goals,\nsuch as 'going to a chair'. We also deploy CMP on physical robots in indoor\nenvironments, where it achieves reasonable performance, even though it is\ntrained entirely in simulation.","url_abs":"http://arxiv.org/abs/1702.03920v3","url_pdf":"http://arxiv.org/pdf/1702.03920v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/dongniu0927/cognitive_mapping_and_planning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/jasonzhang929/CMP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/s-gupta/map-plan-baseline","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/tensorflow/models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/tensorflow/models/tree/master/research/cognitive_mapping_and_planning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"cognitive-mapping-and-planning-for-visual","repo_url":"https://github.com/zuoxingdong/VIN_PyTorch_Visdom","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"visual-navigation","task_name":"Visual Navigation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1702.03920","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}