{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-visual-chain-of-thought","title":"Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization","arxiv_id":"2504.18397","date":"2025-04-25","proceeding":null,"authors":["Kesen Zhao","Beier Zhu","Qianru Sun","Hanwang Zhang"],"abstract":"Chain-of-thought (CoT) reasoning greatly improves the interpretability and problem-solving abilities of multimodal large language models (MLLMs). However, existing approaches are focused on text CoT, limiting their ability to leverage visual cues. Visual CoT remains underexplored, and the only work is based on supervised fine-tuning (SFT) that relies on extensive labeled bounding-box data and is hard to generalize to unseen cases. In this paper, we introduce Unsupervised Visual CoT (UV-CoT), a novel framework for image-level CoT reasoning via preference optimization. UV-CoT performs preference comparisons between model-generated bounding boxes (one is preferred and the other is dis-preferred), eliminating the need for bounding-box annotations. We get such preference data by introducing an automatic data generation pipeline. Given an image, our target MLLM (e.g., LLaVA-1.5-7B) generates seed bounding boxes using a template prompt and then answers the question using each bounded region as input. An evaluator MLLM (e.g., OmniLLM-12B) ranks the responses, and these rankings serve as supervision to train the target MLLM with UV-CoT by minimizing negative log-likelihood losses. By emulating human perception--identifying key regions and reasoning based on them--UV-CoT can improve visual comprehension, particularly in spatial reasoning tasks where textual descriptions alone fall short. Our experiments on six datasets demonstrate the superiority of UV-CoT, compared to the state-of-the-art textual and visual CoT methods. Our zero-shot testing on four unseen datasets shows the strong generalization of UV-CoT. The code is available in https://github.com/kesenzhao/UV-CoT.","url_abs":"https://arxiv.org/abs/2504.18397v1","url_pdf":"https://arxiv.org/pdf/2504.18397v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-visual-chain-of-thought","repo_url":"https://github.com/kesenzhao/uv-cot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.18397","atlas_url":"https://app.syntology.ai/?focus=2504.18397","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.18397"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kesenzhao/uv-cot","reach":null}],"summary":{"ran":3,"ran_fixture":11,"ran_violates":3,"ran_draft_wrong":7,"ran_honours":1,"unverified":5},"by_repo_kind":{"official":{"samples":29,"ran":24,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":30,"samples":[{"code_sha256_prefix":"5be6156a5bbf6aa8","entry":"Identity","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5be6156a5bbf6aa8"}},{"code_sha256_prefix":"93a39f7d4d0d010b","entry":"OmniLMMConfig","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"93a39f7d4d0d010b"}},{"code_sha256_prefix":"5a33fedd425592c9","entry":"Resampler","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a33fedd425592c9"}},{"code_sha256_prefix":"a95e6290546a4d45","entry":"autocontrast_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a95e6290546a4d45"}},{"code_sha256_prefix":"927d3870a6db2930","entry":"brightness_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"927d3870a6db2930"}},{"code_sha256_prefix":"3b8ecf5870557683","entry":"color_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b8ecf5870557683"}},{"code_sha256_prefix":"dd1d12a456bbf084","entry":"contrast_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dd1d12a456bbf084"}},{"code_sha256_prefix":"8c2baaccb692ba62","entry":"enhance_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c2baaccb692ba62"}},{"code_sha256_prefix":"eb4f4c5587a11a2c","entry":"equalize_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb4f4c5587a11a2c"}},{"code_sha256_prefix":"50c4398a8645438a","entry":"get_abs_pos","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"50c4398a8645438a"}},{"code_sha256_prefix":"1dd2ab205689d45b","entry":"identity_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1dd2ab205689d45b"}},{"code_sha256_prefix":"e8b6cf1e9387fc9a","entry":"mean","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e8b6cf1e9387fc9a"}},{"code_sha256_prefix":"ffd0bebcc0b9c875","entry":"none_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffd0bebcc0b9c875"}},{"code_sha256_prefix":"0addd1e67c37549d","entry":"posterize_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0addd1e67c37549d"}},{"code_sha256_prefix":"1a262e5956a7dffb","entry":"rotate_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a262e5956a7dffb"}},{"code_sha256_prefix":"70c60b9689a84a12","entry":"rotate_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70c60b9689a84a12"}},{"code_sha256_prefix":"a0970531e41dd9dd","entry":"sharpness_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0970531e41dd9dd"}},{"code_sha256_prefix":"04fcf271841414c8","entry":"shear_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04fcf271841414c8"}},{"code_sha256_prefix":"37929a3615a6c868","entry":"shear_x_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37929a3615a6c868"}},{"code_sha256_prefix":"bc87971adee63b03","entry":"shear_y_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bc87971adee63b03"}},{"code_sha256_prefix":"f00b29ccb5c157cd","entry":"solarize_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f00b29ccb5c157cd"}},{"code_sha256_prefix":"cd79a8d919027e2d","entry":"solarize_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd79a8d919027e2d"}},{"code_sha256_prefix":"6c41b36881cba094","entry":"translate_level_to_args","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c41b36881cba094"}},{"code_sha256_prefix":"8fdbb5c32fbaadd3","entry":"translate_x_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8fdbb5c32fbaadd3"}},{"code_sha256_prefix":"77c33034a7472197","entry":"translate_y_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77c33034a7472197"}},{"code_sha256_prefix":"70d86725d7cc8930","entry":"OmniLMMModel","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70d86725d7cc8930"}},{"code_sha256_prefix":"607f6d81c5fbcd61","entry":"RandomAugment","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"607f6d81c5fbcd61"}},{"code_sha256_prefix":"ce385361217c109c","entry":"build_transform","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce385361217c109c"}},{"code_sha256_prefix":"18b789fcee7f1fac","entry":"create_vision_module","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18b789fcee7f1fac"}},{"code_sha256_prefix":"e3a9bbe286eaad72","entry":"posterize_func","repo":"kesenzhao/uv-cot","repo_kind":"official","path":"omnilmm/model/omnilmm.py","file_url":"https://github.com/kesenzhao/uv-cot/blob/HEAD/omnilmm/model/omnilmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3a9bbe286eaad72"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}