{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/glus-global-local-reasoning-unified-into-a","title":"GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation","arxiv_id":"2504.07962","date":"2025-04-10","proceeding":"CVPR 2025 1","authors":["Lang Lin","Xueyang Yu","Ziqi Pang","Yu-Xiong Wang"],"abstract":"This paper proposes a novel framework utilizing multi-modal large language models (MLLMs) for referring video object segmentation (RefVOS). Previous MLLM-based methods commonly struggle with the dilemma between \"Ref\" and \"VOS\": they either specialize in understanding a few key frames (global reasoning) or tracking objects on continuous frames (local reasoning), and rely on external VOS or frame selectors to mitigate the other end of the challenge. However, our framework GLUS shows that global and local consistency can be unified into a single video segmentation MLLM: a set of sparse \"context frames\" provides global information, while a stream of continuous \"query frames\" conducts local object tracking. This is further supported by jointly training the MLLM with a pre-trained VOS memory bank to simultaneously digest short-range and long-range temporal information. To improve the information efficiency within the limited context window of MLLMs, we introduce object contrastive learning to distinguish hard false-positive objects and a self-refined framework to identify crucial frames and perform propagation. By collectively integrating these insights, our GLUS delivers a simple yet effective baseline, achieving new state-of-the-art for MLLMs on the MeViS and Ref-Youtube-VOS benchmark. Our project page is at https://glus-video.github.io/.","url_abs":"https://arxiv.org/abs/2504.07962v1","url_pdf":"https://arxiv.org/pdf/2504.07962v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"glus-global-local-reasoning-unified-into-a","repo_url":"https://github.com/GLUS-video/GLUS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"referring-video-object-segmentation","task_name":"Referring Video Object Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"set","method_name":"SET"},{"method_slug":"vos","method_name":"VOS"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/referring-video-object-segmentation-on-long","task":"Referring Video Object Segmentation","dataset":"Long-RVOS","model":"GLUS","rank_in_archive_order":4,"of":7,"metrics":{"J&F":"36.6","tIoU":"68.4","vIoU":"34.6"},"uses_additional_data":false},{"leaderboard":"/sota/referring-video-object-segmentation-on-mevis","task":"Referring Video Object Segmentation","dataset":"MeViS","model":"GLUS","rank_in_archive_order":3,"of":16,"metrics":{"F":"54.2","J":"48.5","J&F":"51.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.07962","atlas_url":"https://app.syntology.ai/?focus=2504.07962","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.07962"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GLUS-video/GLUS","reach":null}],"summary":{"ran":3,"unverified":9},"by_repo_kind":{"listed":{"samples":12,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"fafc49e646cbb27e","entry":"ContrastiveLoss","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fafc49e646cbb27e"}},{"code_sha256_prefix":"fd826b7940bc9b38","entry":"LlavaLlamaModel","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd826b7940bc9b38"}},{"code_sha256_prefix":"c07b53902191d016","entry":"segTokenBank","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c07b53902191d016"}},{"code_sha256_prefix":"f336c18e86b8d8a3","entry":"GLUSForCausalLM","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f336c18e86b8d8a3"}},{"code_sha256_prefix":"78b6a27c210c8801","entry":"GlusMetaModel","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78b6a27c210c8801"}},{"code_sha256_prefix":"5cfd95bfb0a02aaa","entry":"GlusModel","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5cfd95bfb0a02aaa"}},{"code_sha256_prefix":"b77d0b4e10b306e7","entry":"LlavaLlamaForCausalLM","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b77d0b4e10b306e7"}},{"code_sha256_prefix":"65e399d0eab7c33e","entry":"LlavaMetaForCausalLM","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65e399d0eab7c33e"}},{"code_sha256_prefix":"9fb467589b9d20f5","entry":"LlavaMetaModel","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9fb467589b9d20f5"}},{"code_sha256_prefix":"3f123a3de662ff17","entry":"SAM2Transforms","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f123a3de662ff17"}},{"code_sha256_prefix":"9ad65f1c27158aa9","entry":"_load_checkpoint","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ad65f1c27158aa9"}},{"code_sha256_prefix":"eb1cf9ae06cf7744","entry":"build_sam2","repo":"GLUS-video/GLUS","repo_kind":"listed","path":"model/GLUS.py","file_url":"https://github.com/GLUS-video/GLUS/blob/HEAD/model/GLUS.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb1cf9ae06cf7744"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}