{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/find-what-you-want-learning-demand-1","title":"Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation","arxiv_id":"2309.08138","date":"2023-09-15","proceeding":"NeurIPS 2023 11","authors":["Hongcheng Wang","Andy Guan Hong Chen","Xiaoqi Li","Mingdong Wu","Hao Dong"],"abstract":"The task of Visual Object Navigation (VON) involves an agent's ability to locate a particular object within a given scene. In order to successfully accomplish the VON task, two essential conditions must be fulfilled:1) the user must know the name of the desired object; and 2) the user-specified object must actually be present within the scene. To meet these conditions, a simulator can incorporate pre-defined object names and positions into the metadata of the scene. However, in real-world scenarios, it is often challenging to ensure that these conditions are always met. Human in an unfamiliar environment may not know which objects are present in the scene, or they may mistakenly specify an object that is not actually present. Nevertheless, despite these challenges, human may still have a demand for an object, which could potentially be fulfilled by other objects present within the scene in an equivalent manner. Hence, we propose Demand-driven Navigation (DDN), which leverages the user's demand as the task instruction and prompts the agent to find the object matches the specified demand. DDN aims to relax the stringent conditions of VON by focusing on fulfilling the user's demand rather than relying solely on predefined object categories or names. We propose a method first acquire textual attribute features of objects by extracting common knowledge from a large language model. These textual attribute features are subsequently aligned with visual attribute features using Contrastive Language-Image Pre-training (CLIP). By incorporating the visual attribute features as prior knowledge, we enhance the navigation process. Experiments on AI2Thor with the ProcThor dataset demonstrate the visual attribute features improve the agent's navigation performance and outperform the baseline methods commonly used in VON.","url_abs":"https://arxiv.org/abs/2309.08138v3","url_pdf":"https://arxiv.org/pdf/2309.08138v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"find-what-you-want-learning-demand-1","repo_url":"https://github.com/whcpumpkin/demand-driven-navigation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.08138","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.08138"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/whcpumpkin/demand-driven-navigation","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"e14269a86b6d505a","entry":"Categorical","repo":"whcpumpkin/demand-driven-navigation","repo_kind":"official","path":"agent.py","file_url":"https://github.com/whcpumpkin/demand-driven-navigation/blob/HEAD/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e14269a86b6d505a"}},{"code_sha256_prefix":"cf1c3322c38f812c","entry":"attention_attribute_model","repo":"whcpumpkin/demand-driven-navigation","repo_kind":"official","path":"agent.py","file_url":"https://github.com/whcpumpkin/demand-driven-navigation/blob/HEAD/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cf1c3322c38f812c"}},{"code_sha256_prefix":"a8c8289945598975","entry":"fusion_model","repo":"whcpumpkin/demand-driven-navigation","repo_kind":"official","path":"agent.py","file_url":"https://github.com/whcpumpkin/demand-driven-navigation/blob/HEAD/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8c8289945598975"}},{"code_sha256_prefix":"8496ce0c36c00dcf","entry":"get_dataloader","repo":"whcpumpkin/Demand-driven-navigation","repo_kind":"official","path":"train_attribute_features.py","file_url":"https://github.com/whcpumpkin/Demand-driven-navigation/blob/HEAD/train_attribute_features.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8496ce0c36c00dcf"}},{"code_sha256_prefix":"8a5773c591ee62bd","entry":"Agent","repo":"whcpumpkin/demand-driven-navigation","repo_kind":"official","path":"agent.py","file_url":"https://github.com/whcpumpkin/demand-driven-navigation/blob/HEAD/agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a5773c591ee62bd"}},{"code_sha256_prefix":"64a642d0b4735a01","entry":"BaseModel","repo":"whcpumpkin/demand-driven-navigation","repo_kind":"official","path":"agent.py","file_url":"https://github.com/whcpumpkin/demand-driven-navigation/blob/HEAD/agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64a642d0b4735a01"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}