{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-26142","title":"Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentation","arxiv_id":"2608.26142","date":"2026-06-26","proceeding":"ICML","authors":["Yuhan Liu","Yixiong Zou","Yuhua Li","Ruixuan Li"],"abstract":"Referring Expression Segmentation (RES) aims to generate pixel-wise segmentation masks from complex and implicit textual queries. While recent advances in Multimodal Large Language Models (MLLMs) have substantially boosted RES performance, their prohibitive computational overhead remains a critical bottleneck, which, however, is rarely explored. To fill this gap, we first evaluate typical token compression methods on this task and observe a surprising performance degradation. In this paper, we aim to understand this phenomenon for a solution. By extensive experiments, we find that token compression for RES requires preserving the original position embeddings and local neighboring spatial structures, indicating that visual token position information is far more critical than in other tasks. Building on this insight, we ask: Can we design the token compression method purely based on the position information? Therefore, we propose PAYN, a plug-and-play, training-free token compression method that relies solely on position information. PAYN retains tokens that are adequately distributed in every local neighboring region while strictly preserving original positional indices, thereby maintaining spatial relational consistency. Experiments on multiple RES benchmarks demonstrate that our method outperforms existing token compression methods, verifying that position is indeed all you need for token compression in the MLLM-based RES task. Codes are avaliable at https://github.com/YuhanLiu231/PAYN.","url_abs":"https://arxiv.org/abs/2608.26142","url_pdf":"https://arxiv.org/pdf/2608.26142","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2608.26142","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.26142"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/YuhanLiu231/PAYN","reach":null}],"summary":{"ran_draft_wrong":3,"ran_fixture":1,"ran_honours":1,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"5950b7de426c2b29","entry":"CLIPVisionTower_payn_forward","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5950b7de426c2b29"}},{"code_sha256_prefix":"f14b2214e927b61a","entry":"LlavaForConditionalGeneration_get_image_features_payn","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f14b2214e927b61a"}},{"code_sha256_prefix":"2b5bd5fb0350d5eb","entry":"fix_position_embeddings","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2b5bd5fb0350d5eb"}},{"code_sha256_prefix":"bb967b4a649cc8e1","entry":"fix_position_ids","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bb967b4a649cc8e1"}},{"code_sha256_prefix":"cc2040f43dcdaa76","entry":"get_fps_indices","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cc2040f43dcdaa76"}},{"code_sha256_prefix":"92b97f0d02c9af24","entry":"LlamaModel_forward_payn","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"92b97f0d02c9af24"}},{"code_sha256_prefix":"78bca3853b2754e6","entry":"LlavaForConditionalGeneration_forward_payn","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"78bca3853b2754e6"}},{"code_sha256_prefix":"5818c3ac1fd17e5c","entry":"llava_payn","repo":"YuhanLiu231/PAYN","repo_kind":"found_in_text","path":"ms-swift/swift/payn/compression.py","file_url":"https://github.com/YuhanLiu231/PAYN/blob/HEAD/ms-swift/swift/payn/compression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5818c3ac1fd17e5c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}