{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-dual-semantic-aware-recurrent-global","title":"A Dual Semantic-Aware Recurrent Global-Adaptive Network For Vision-and-Language Navigation","arxiv_id":"2305.03602","date":"2023-05-05","proceeding":null,"authors":["Liuyi Wang","Zongtao He","Jiagui Tang","Ronghao Dang","Naijia Wang","Chengju Liu","Qijun Chen"],"abstract":"Vision-and-Language Navigation (VLN) is a realistic but challenging task that requires an agent to locate the target region using verbal and visual cues. While significant advancements have been achieved recently, there are still two broad limitations: (1) The explicit information mining for significant guiding semantics concealed in both vision and language is still under-explored; (2) The previously structured map method provides the average historical appearance of visited nodes, while it ignores distinctive contributions of various images and potent information retention in the reasoning process. This work proposes a dual semantic-aware recurrent global-adaptive network (DSRG) to address the above problems. First, DSRG proposes an instruction-guidance linguistic module (IGL) and an appearance-semantics visual module (ASV) for boosting vision and language semantic learning respectively. For the memory mechanism, a global adaptive aggregation module (GAA) is devised for explicit panoramic observation fusion, and a recurrent memory fusion module (RMF) is introduced to supply implicit temporal hidden states. Extensive experimental results on the R2R and REVERIE datasets demonstrate that our method achieves better performance than existing methods. Code is available at https://github.com/CrystalSixone/DSRG.","url_abs":"https://arxiv.org/abs/2305.03602v2","url_pdf":"https://arxiv.org/pdf/2305.03602v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-dual-semantic-aware-recurrent-global","repo_url":"https://github.com/crystalsixone/dsrg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"vision-and-language-navigation","task_name":"Vision and Language Navigation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.03602","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.03602"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/CrystalSixone/DSRG","reach":null}],"summary":{"ran":6,"ran_draft_wrong":4,"unverified":14},"by_repo_kind":{"official":{"samples":24,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4bdbd317043aa0de","entry":"AggregateAttention","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4bdbd317043aa0de"}},{"code_sha256_prefix":"cd1fc93df76d6f16","entry":"BertIntermediate","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd1fc93df76d6f16"}},{"code_sha256_prefix":"bfb5cdf1a9ced435","entry":"BertOutAttention","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfb5cdf1a9ced435"}},{"code_sha256_prefix":"ffff1ac5549ddff4","entry":"BertSelfAttention","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffff1ac5549ddff4"}},{"code_sha256_prefix":"048b3530e2f008f1","entry":"TransformerEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"048b3530e2f008f1"}},{"code_sha256_prefix":"fac115820b1e6391","entry":"TransformerEncoderLayer","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac115820b1e6391"}},{"code_sha256_prefix":"698f5d9bbd2a4581","entry":"_get_clones","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"698f5d9bbd2a4581"}},{"code_sha256_prefix":"8b70f6a5cf504379","entry":"create_transformer_encoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b70f6a5cf504379"}},{"code_sha256_prefix":"455413c9b171cfe7","entry":"extend_neg_masks","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"455413c9b171cfe7"}},{"code_sha256_prefix":"68ae8be3692cbbbe","entry":"gen_seq_masks","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68ae8be3692cbbbe"}},{"code_sha256_prefix":"b38b941722e63d29","entry":"BertAttention","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b38b941722e63d29"}},{"code_sha256_prefix":"887a39ce2142a57d","entry":"BertEmbeddings","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"887a39ce2142a57d"}},{"code_sha256_prefix":"1ae89ecd5c078970","entry":"BertEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ae89ecd5c078970"}},{"code_sha256_prefix":"c213722ebbdfaf05","entry":"BertLayer","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c213722ebbdfaf05"}},{"code_sha256_prefix":"4dc8cae13c724e85","entry":"BertPooler","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4dc8cae13c724e85"}},{"code_sha256_prefix":"6bed46b39566e949","entry":"BertXAttention","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6bed46b39566e949"}},{"code_sha256_prefix":"c526eb56d5c9d584","entry":"ClsPrediction","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c526eb56d5c9d584"}},{"code_sha256_prefix":"786a07db08c7dd5b","entry":"CrossmodalEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"786a07db08c7dd5b"}},{"code_sha256_prefix":"298074699857e928","entry":"GlobalMapEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"298074699857e928"}},{"code_sha256_prefix":"a36464347726e6de","entry":"GlocalTextPathNavCMT","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a36464347726e6de"}},{"code_sha256_prefix":"f0ad0689537a8d37","entry":"GraphLXRTXLayer","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0ad0689537a8d37"}},{"code_sha256_prefix":"7c9d8a3caac3b3b7","entry":"ImageEmbeddings","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c9d8a3caac3b3b7"}},{"code_sha256_prefix":"f108f9f644e26f09","entry":"LanguageEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f108f9f644e26f09"}},{"code_sha256_prefix":"2dd5230398b073b5","entry":"LocalVPEncoder","repo":"CrystalSixone/DSRG","repo_kind":"official","path":"finetune_src/models/vilmodel.py","file_url":"https://github.com/CrystalSixone/DSRG/blob/HEAD/finetune_src/models/vilmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2dd5230398b073b5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}