{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-visual-grounding-a-survey","title":"Towards Visual Grounding: A Survey","arxiv_id":"2412.20206","date":"2024-12-28","proceeding":null,"authors":["Linhui Xiao","Xiaoshan Yang","Xiangyuan Lan","YaoWei Wang","Changsheng Xu"],"abstract":"Visual Grounding is also known as Referring Expression Comprehension and Phrase Grounding. It involves localizing a natural number of specific regions within an image based on a given textual description. The objective of this task is to emulate the prevalent referential relationships in social conversations, equipping machines with human-like multimodal comprehension capabilities. Consequently, it has extensive applications in various domains. However, since 2021, visual grounding has witnessed significant advancements, with emerging new concepts such as grounded pre-training, grounding multimodal LLMs, generalized visual grounding, and giga-pixel grounding, which have brought numerous new challenges. In this survey, we initially examine the developmental history of visual grounding and provide an overview of essential background knowledge. We systematically track and summarize the advancements and meticulously organize the various settings in visual grounding, thereby establishing precise definitions of these settings to standardize future research and ensure a fair comparison. Additionally, we delve into several advanced topics and highlight numerous applications of visual grounding. Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers. By extracting common technical details, this survey encompasses the representative works in each subtopic over the past decade. To the best, this paper presents the most comprehensive overview currently available in the field of grounding. This survey is designed to be suitable for both beginners and experienced researchers, serving as an invaluable resource for understanding key concepts and tracking the latest research developments. We keep tracing related works at https://github.com/linhuixiao/Awesome-Visual-Grounding.","url_abs":"https://arxiv.org/abs/2412.20206v1","url_pdf":"https://arxiv.org/pdf/2412.20206v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-visual-grounding-a-survey","repo_url":"https://github.com/linhuixiao/awesome-visual-grounding","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-visual-grounding-a-survey","repo_url":"https://github.com/linhuixiao/clip-vg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-visual-grounding-a-survey","repo_url":"https://github.com/linhuixiao/hivg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"towards-visual-grounding-a-survey","repo_url":"https://github.com/linhuixiao/oneref","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"phrase-grounding","task_name":"Phrase Grounding"},{"task_slug":"referring-expression","task_name":"Referring Expression"},{"task_slug":"referring-expression-comprehension","task_name":"Referring Expression Comprehension"},{"task_slug":"survey","task_name":"Survey"},{"task_slug":"visual-grounding","task_name":"Visual Grounding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.20206","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.20206"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linhuixiao/awesome-visual-grounding","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linhuixiao/hivg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linhuixiao/oneref","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linhuixiao/clip-vg","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":6,"ran":3,"unverified":6},"by_repo_kind":{"listed":{"samples":12,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"e33b112514a9e4b0","entry":"generate2","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/predict.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/predict.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e33b112514a9e4b0"}},{"code_sha256_prefix":"883320eb62f15d12","entry":"generate_beam","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning_for_dataset.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning_for_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"883320eb62f15d12"}},{"code_sha256_prefix":"e1fe6b1f19460a65","entry":"get_device","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e1fe6b1f19460a65"}},{"code_sha256_prefix":"4a9deb3a2060d602","entry":"get_object_phrase","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4a9deb3a2060d602"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"596ccf0324e358f5","entry":"get_predicate_phrase","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"596ccf0324e358f5"}},{"code_sha256_prefix":"c366b440880ba949","entry":"get_subject_phrase","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c366b440880ba949"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"c68cb2401825d6a1","entry":"add_period","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c68cb2401825d6a1"}},{"code_sha256_prefix":"c9a8ee28fc96c79b","entry":"build_model","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"models/clip/model.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/models/clip/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c9a8ee28fc96c79b"}},{"code_sha256_prefix":"37664f80d7c18871","entry":"generate2","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning_for_dataset.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/clip_prefix_captioning_for_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37664f80d7c18871"}},{"code_sha256_prefix":"0a286fb23ba005b9","entry":"get_image","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a286fb23ba005b9"}},{"code_sha256_prefix":"b91b981015bbf498","entry":"load","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"models/clip/clip.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/models/clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b91b981015bbf498"}},{"code_sha256_prefix":"50449115c68a0ddb","entry":"thread","repo":"linhuixiao/clip-vg","repo_kind":"listed","path":"pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","file_url":"https://github.com/linhuixiao/clip-vg/blob/HEAD/pseudo_label_generation_module/pseudo_caption_label_generation/CLIP_prefix_caption/parse_conceptual.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50449115c68a0ddb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}