{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hop-history-and-order-aware-pre-training-for","title":"HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation","arxiv_id":"2203.11591","date":"2022-03-22","proceeding":"CVPR 2022 1","authors":["Yanyuan Qiao","Yuankai Qi","Yicong Hong","Zheng Yu","Peng Wang","Qi Wu"],"abstract":"Pre-training has been adopted in a few of recent works for Vision-and-Language Navigation (VLN). However, previous pre-training methods for VLN either lack the ability to predict future actions or ignore the trajectory contexts, which are essential for a greedy navigation process. In this work, to promote the learning of spatio-temporal visual-textual correspondence as well as the agent's capability of decision making, we propose a novel history-and-order aware pre-training paradigm (HOP) with VLN-specific objectives that exploit the past observations and support future action prediction. Specifically, in addition to the commonly used Masked Language Modeling (MLM) and Trajectory-Instruction Matching (TIM), we design two proxy tasks to model temporal order information: Trajectory Order Modeling (TOM) and Group Order Modeling (GOM). Moreover, our navigation action prediction is also enhanced by introducing the task of Action Prediction with History (APH), which takes into account the history visual perceptions. Extensive experimental results on four downstream VLN tasks (R2R, REVERIE, NDH, RxR) demonstrate the effectiveness of our proposed method compared against several state-of-the-art agents.","url_abs":"https://arxiv.org/abs/2203.11591v1","url_pdf":"https://arxiv.org/pdf/2203.11591v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hop-history-and-order-aware-pre-training-for","repo_url":"https://github.com/yanyuanqiao/hop-vln","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"masked-language-modeling","task_name":"Masked Language Modeling"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"vision-and-language-navigation","task_name":"Vision and Language Navigation"},{"task_slug":"visual-navigation","task_name":"Visual Navigation"}],"methods":[{"method_slug":"aware","method_name":"AWARE"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-navigation-on-room-to-room-1","task":"Visual Navigation","dataset":"R2R","model":"HOP","rank_in_archive_order":5,"of":11,"metrics":{"spl":"0.59"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.11591","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.11591"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yanyuanqiao/hop-vln","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_honours":1,"unverified":8},"by_repo_kind":{"official":{"samples":12,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/vilmodel.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/vilmodel.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"fdc64f4c72036ae4","entry":"gelu","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/vilmodel.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/vilmodel.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fdc64f4c72036ae4"}},{"code_sha256_prefix":"0c145728a35789fa","entry":"load_tf_weights_in_bert","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/vilmodel.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/vilmodel.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c145728a35789fa"}},{"code_sha256_prefix":"d30d6c3098df2c42","entry":"prune_linear_layer","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/modeling_utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d30d6c3098df2c42"}},{"code_sha256_prefix":"77b696d33372b6d9","entry":"build_viewpoint_loc_embedding","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/batch_loader.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/batch_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77b696d33372b6d9"}},{"code_sha256_prefix":"640883dda75a7108","entry":"build_viewpoint_loc_embedding_1","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/batch_loader.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/batch_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"640883dda75a7108"}},{"code_sha256_prefix":"773c877d3719d36d","entry":"build_vocab","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"773c877d3719d36d"}},{"code_sha256_prefix":"bd7ea3e7a698a3e0","entry":"load_datasets","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd7ea3e7a698a3e0"}},{"code_sha256_prefix":"e3a90df1a875a38e","entry":"load_nav_graphs","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3a90df1a875a38e"}},{"code_sha256_prefix":"a66970e935c65b3d","entry":"new_mask_tokens","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/batch_loader.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/batch_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a66970e935c65b3d"}},{"code_sha256_prefix":"7183a17dd79327a8","entry":"prune_conv1d_layer","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/modeling_utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7183a17dd79327a8"}},{"code_sha256_prefix":"e31093ffc8275cff","entry":"prune_layer","repo":"yanyuanqiao/hop-vln","repo_kind":"official","path":"tasks/pretrain/modeling_utils.py","file_url":"https://github.com/yanyuanqiao/hop-vln/blob/HEAD/tasks/pretrain/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e31093ffc8275cff"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}