{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-03853","title":"UGround: Towards Unified Visual Grounding with Unrolled Transformers","arxiv_id":"2510.03853","date":"2025-10-04","proceeding":"ICML","authors":["Rui Qian","Xin Yin","Chuanhang Deng","Zhiyuan Peng","Jian Xiong","Wei Zhai","Dejing Dou"],"abstract":"We present UGround, a \\textbf{U}nified visual \\textbf{Ground}ing paradigm that dynamically selects intermediate layers across \\textbf{U}nrolled transformers as ``mask as prompt,'' diverging from the prevailing pipeline that leverages the fixed last hidden layer as ``\\texttt{<SEG>} as prompt.'' UGround addresses two primary challenges posed by the prevailing paradigm: (1) its reliance on the fixed last hidden layer, which sequentially amplifies cumulative errors arising from layer-by-layer propagation without intermediate correction, and (2) its use of \\texttt{<SEG>} as a prompt, which implicitly projects textual embeddings into visual space without explicit spatial cues (e.g., coordinates). Central to UGround is Policy-Prompted Masking, which comprises two key components: Stochastic Skip Connection (SSC) and Mask as Prompt (MasP). SSC is a reinforcement learning policy that, via stochastic sampling, allows each \\texttt{<SEG>} token to slide across unrolled transformer layers, enabling dynamic layer selection at which it connects to the vision model (e.g., SAM) in a skip-connection fashion. Given the selected hidden layer, MasP uses the similarity map derived from the \\texttt{<SEG>} token and image tokens as a soft logit mask to prompt SAM for mask generation, offering explicit spatial cues through its activation regions. To validate the effectiveness of UGround, we, for the first time, have unified visual grounding within a single framework from an attribute perspective, spanning from traditional refer expression segmentation to newly proposed reasoning segmentation, single-target to multi-target, positive query to false premise (empty target). All code and models are publicly available at https://github.com/rui-qian/UGround.","url_abs":"https://arxiv.org/abs/2510.03853","url_pdf":"https://arxiv.org/pdf/2510.03853","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2510.03853","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.03853"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ruiqian/UGround","reach":{"status":"gone","observed_at":"2026-09-16","how":"tree_404+repo_404"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/rui-qian/UGround","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"found_in_text":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a1c2083ec2eaeb22","entry":"dice_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/UGround.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/UGround.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a1c2083ec2eaeb22"}},{"code_sha256_prefix":"e098b877e1d246a5","entry":"overlap_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/PixelLM.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/PixelLM.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e098b877e1d246a5"}},{"code_sha256_prefix":"77ff24f1cbcb4f2d","entry":"preprocess","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"chat.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/chat.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"77ff24f1cbcb4f2d"}},{"code_sha256_prefix":"6ec47ecac3fe27fc","entry":"dice_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/GSVA.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/GSVA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ec47ecac3fe27fc"}},{"code_sha256_prefix":"62c7b02d97a929b3","entry":"get_model_config","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"configs/model_config.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/configs/model_config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62c7b02d97a929b3"}},{"code_sha256_prefix":"af6fc3d42a9de09a","entry":"iou_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/GSVA.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/GSVA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af6fc3d42a9de09a"}},{"code_sha256_prefix":"bfb6ea6413f9abf5","entry":"parse_args","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"merge_lora_weights_and_save_hf_model.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/merge_lora_weights_and_save_hf_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfb6ea6413f9abf5"}},{"code_sha256_prefix":"3face96ac23ae5bc","entry":"sigmoid_ce_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/UGround.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/UGround.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3face96ac23ae5bc"}},{"code_sha256_prefix":"13c0bb91f8009d9c","entry":"sigmoid_ce_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/GSVA.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/GSVA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"13c0bb91f8009d9c"}},{"code_sha256_prefix":"5e032c86b35b7a19","entry":"sigmoid_focal_loss","repo":"rui-qian/UGround","repo_kind":"found_in_text","path":"model/READ.py","file_url":"https://github.com/rui-qian/UGround/blob/HEAD/model/READ.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e032c86b35b7a19"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}