{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-14374","title":"Spatial Preference Rewarding for MLLMs Spatial Understanding","arxiv_id":"2510.14374","date":"2025-10-16","proceeding":"ICCV","authors":["Han Qiu","Peng Gao","Lewei Lu","Xiaoqin Zhang","Ling Shao","Shijian Lu"],"abstract":"Multimodal large language models~(MLLMs) have demonstrated promising spatial understanding capabilities, such as referencing and grounding object descriptions. Despite their successes, MLLMs still fall short in fine-grained spatial perception abilities, such as generating detailed region descriptions or accurately localizing objects. Additionally, they often fail to respond to the user's requirements for desired fine-grained spatial understanding. This issue might arise because existing approaches primarily focus on tuning MLLMs to model pre-annotated instruction data to inject spatial knowledge, without direct supervision of MLLMs' actual responses. We address this issue by SPR, a Spatial Preference Rewarding~(SPR) approach that enhances MLLMs' spatial capabilities by rewarding MLLMs' detailed responses with precise object localization over vague or inaccurate responses. With randomly selected image regions and region descriptions from MLLMs, SPR introduces semantic and localization scores to comprehensively evaluate the text quality and localization quality in MLLM-generated descriptions. We also refine the MLLM descriptions with better localization accuracy and pair the best-scored refinement with the initial descriptions of the lowest score for direct preference optimization, thereby enhancing fine-grained alignment with visual input. Extensive experiments over standard referring and grounding benchmarks show that SPR improves MLLM spatial understanding capabilities effectively with minimal overhead in training. Data and code will be released at https://github.com/hanqiu-hq/SPR","url_abs":"https://arxiv.org/abs/2510.14374","url_pdf":"https://arxiv.org/pdf/2510.14374","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2510.14374","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.14374"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/hanqiu-hq/SPR","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran_draft_wrong":3,"unverified":13},"by_repo_kind":{"found_in_text":{"samples":17,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"d515c6ab54f0598c","entry":"are_phrases_similar","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/eval_flickr_entities.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/eval_flickr_entities.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d515c6ab54f0598c"}},{"code_sha256_prefix":"42a46570620cd9fa","entry":"get_chunk","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/ferret_score_siglip.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/ferret_score_siglip.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42a46570620cd9fa"}},{"code_sha256_prefix":"5a8ea802e3fdbf73","entry":"resize_bbox","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/eval_flickr_entities.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/eval_flickr_entities.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a8ea802e3fdbf73"}},{"code_sha256_prefix":"076c252c52cbb161","entry":"split_list","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/ferret_score_siglip.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/ferret_score_siglip.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"076c252c52cbb161"}},{"code_sha256_prefix":"9b249cdc1ef5003c","entry":"crop_local_region","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/image_process.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/image_process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b249cdc1ef5003c"}},{"code_sha256_prefix":"693a88c32834ebef","entry":"crop_square_image","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/image_process.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/image_process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"693a88c32834ebef"}},{"code_sha256_prefix":"630a78a1ce5d571d","entry":"decode_bbox_from_caption","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/eval_flickr_entities.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/eval_flickr_entities.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"630a78a1ce5d571d"}},{"code_sha256_prefix":"b8c3dbc07ce34196","entry":"decode_bbox_from_caption","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/eval_refexp.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/eval_refexp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8c3dbc07ce34196"}},{"code_sha256_prefix":"d036330f8a6a9a72","entry":"extract_boxes_from_text","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/ground_utils.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/ground_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d036330f8a6a9a72"}},{"code_sha256_prefix":"279b92344bb2cac9","entry":"get_bbox_coor","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"inference/ferret/generate_region_descriptions.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/inference/ferret/generate_region_descriptions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"279b92344bb2cac9"}},{"code_sha256_prefix":"b49928f7e85dc0c6","entry":"map_box_to_origin_text","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/process_text.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/process_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b49928f7e85dc0c6"}},{"code_sha256_prefix":"d26545e5ad9c6fcf","entry":"map_crop_coord_to_origin","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/process_text.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/process_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d26545e5ad9c6fcf"}},{"code_sha256_prefix":"cc4da8158a810e05","entry":"plot_refexp","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/model_refcoco.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/model_refcoco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc4da8158a810e05"}},{"code_sha256_prefix":"4e4baed5bbbfba25","entry":"remove_grounding_text","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/process_text.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/process_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e4baed5bbbfba25"}},{"code_sha256_prefix":"f65f7ca4ffcd4fa6","entry":"remove_punctuation","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"eval/ferret/model_flickr.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/eval/ferret/model_flickr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f65f7ca4ffcd4fa6"}},{"code_sha256_prefix":"5c4c913b24137e65","entry":"split_into_chunks","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/ferret_score_siglip.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/ferret_score_siglip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c4c913b24137e65"}},{"code_sha256_prefix":"c68033b87d69837c","entry":"vision_model_head_forward","repo":"hanqiu-hq/SPR","repo_kind":"found_in_text","path":"construct_data/utils/region_siglip.py","file_url":"https://github.com/hanqiu-hq/SPR/blob/HEAD/construct_data/utils/region_siglip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c68033b87d69837c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}