{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prvql-progressive-knowledge-guided-refinement","title":"PRVQL: Progressive Knowledge-guided Refinement for Robust Egocentric Visual Query Localization","arxiv_id":"2502.07707","date":"2025-02-11","proceeding":null,"authors":["Bing Fan","Yunhe Feng","Yapeng Tian","Yuewei Lin","Yan Huang","Heng Fan"],"abstract":"Egocentric visual query localization (EgoVQL) focuses on localizing the target of interest in space and time from first-person videos, given a visual query. Despite recent progressive, existing methods often struggle to handle severe object appearance changes and cluttering background in the video due to lacking sufficient target cues, leading to degradation. Addressing this, we introduce PRVQL, a novel Progressive knowledge-guided Refinement framework for EgoVQL. The core is to continuously exploit target-relevant knowledge directly from videos and utilize it as guidance to refine both query and video features for improving target localization. Our PRVQL contains multiple processing stages. The target knowledge from one stage, comprising appearance and spatial knowledge extracted via two specially designed knowledge learning modules, are utilized as guidance to refine the query and videos features for the next stage, which are used to generate more accurate knowledge for further feature refinement. With such a progressive process, target knowledge in PRVQL can be gradually improved, which, in turn, leads to better refined query and video features for localization in the final stage. Compared to previous methods, our PRVQL, besides the given object cues, enjoys additional crucial target information from a video as guidance to refine features, and hence enhances EgoVQL in complicated scenes. In our experiments on challenging Ego4D, PRVQL achieves state-of-the-art result and largely surpasses other methods, showing its efficacy. Our code, model and results will be released at https://github.com/fb-reps/PRVQL.","url_abs":"https://arxiv.org/abs/2502.07707v1","url_pdf":"https://arxiv.org/pdf/2502.07707v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prvql-progressive-knowledge-guided-refinement","repo_url":"https://github.com/fb-reps/prvql","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.07707","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.07707"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/fb-reps/PRVQL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"unverified":11},"by_repo_kind":{"official":{"samples":13,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"69e3a274f98c15af","entry":"gelu","repo":"fb-reps/PRVQL","repo_kind":"official","path":"bert_model/bert_module.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/bert_model/bert_module.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69e3a274f98c15af"}},{"code_sha256_prefix":"77601724cad03f95","entry":"gelu_new","repo":"fb-reps/PRVQL","repo_kind":"official","path":"bert_model/bert_module.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/bert_model/bert_module.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77601724cad03f95"}},{"code_sha256_prefix":"e58720b43049fef2","entry":"extract_sub_modules_state_dict","repo":"fb-reps/PRVQL","repo_kind":"official","path":"model/corr_clip_spatial_transformer2_anchor_2heads_hnm.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/model/corr_clip_spatial_transformer2_anchor_2heads_hnm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e58720b43049fef2"}},{"code_sha256_prefix":"fe7c2f09c012c022","entry":"format_predictions","repo":"fb-reps/PRVQL","repo_kind":"official","path":"inference_results.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/inference_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe7c2f09c012c022"}},{"code_sha256_prefix":"070d7eed4f22f51d","entry":"get_bbox_from_data","repo":"fb-reps/PRVQL","repo_kind":"official","path":"dataset/base_dataset.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/dataset/base_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"070d7eed4f22f51d"}},{"code_sha256_prefix":"0e96da7c96574812","entry":"get_losses","repo":"fb-reps/PRVQL","repo_kind":"official","path":"utils/loss_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e96da7c96574812"}},{"code_sha256_prefix":"71ace89be308adbc","entry":"get_losses_head","repo":"fb-reps/PRVQL","repo_kind":"official","path":"utils/loss_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71ace89be308adbc"}},{"code_sha256_prefix":"bed1828c9c920325","entry":"positionalencoding1d","repo":"fb-reps/PRVQL","repo_kind":"official","path":"utils/model_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/utils/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bed1828c9c920325"}},{"code_sha256_prefix":"b2e8731183a22bb6","entry":"positionalencoding2d","repo":"fb-reps/PRVQL","repo_kind":"official","path":"utils/model_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/utils/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2e8731183a22bb6"}},{"code_sha256_prefix":"5b821635ccbae36a","entry":"positionalencoding3d","repo":"fb-reps/PRVQL","repo_kind":"official","path":"utils/model_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/utils/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b821635ccbae36a"}},{"code_sha256_prefix":"af0e3990e7f9effe","entry":"process_data","repo":"fb-reps/PRVQL","repo_kind":"official","path":"dataset/dataset_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/dataset/dataset_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af0e3990e7f9effe"}},{"code_sha256_prefix":"379a559fff316790","entry":"replicate_sample_for_hnm","repo":"fb-reps/PRVQL","repo_kind":"official","path":"dataset/dataset_utils.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/dataset/dataset_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"379a559fff316790"}},{"code_sha256_prefix":"68900fa3cd5db048","entry":"sample_frames_balance","repo":"fb-reps/PRVQL","repo_kind":"official","path":"dataset/base_dataset.py","file_url":"https://github.com/fb-reps/PRVQL/blob/HEAD/dataset/base_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68900fa3cd5db048"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}