{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/feast-your-eyes-mixture-of-resolution","title":"Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models","arxiv_id":"2403.03003","date":"2024-03-05","proceeding":null,"authors":["Gen Luo","Yiyi Zhou","Yuxin Zhang","Xiawu Zheng","Xiaoshuai Sun","Rongrong Ji"],"abstract":"Despite remarkable progress, existing multimodal large language models (MLLMs) are still inferior in granular visual recognition. Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming. Based on this observation, we propose a novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation (MRA). In particular, MRA adopts two visual pathways for images with different resolutions, where high-resolution visual information is embedded into the low-resolution pathway via the novel mixture-of-resolution adapters (MR-Adapters). This design also greatly reduces the input sequence length of MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL) tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g., +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR remain efficient with MRA, e.g., 20 training hours and 3$\\times$ inference speed than LLaVA-1.5. Source codes are released at: https://github.com/luogen1996/LLaVA-HR.","url_abs":"https://arxiv.org/abs/2403.03003v1","url_pdf":"https://arxiv.org/pdf/2403.03003v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"feast-your-eyes-mixture-of-resolution","repo_url":"https://github.com/luogen1996/llava-hr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"TextVQA"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-mm-vet","task":"Visual Question Answering","dataset":"MM-Vet","model":"LLaVA-HR-X","rank_in_archive_order":156,"of":231,"metrics":{"GPT-4 score":"35.5"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2403.03003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.03003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/luogen1996/llava-hr","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":3,"ran":1,"ran_violates":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6a7cafccc873c34d","entry":"evaluate_code","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/model_mathvista.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/model_mathvista.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a7cafccc873c34d"}},{"code_sha256_prefix":"42a46570620cd9fa","entry":"get_chunk","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/model_caption_loader.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/model_caption_loader.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42a46570620cd9fa"}},{"code_sha256_prefix":"ff5aa85ec44db5ee","entry":"normalize_extracted_answer","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/calculate_score.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/calculate_score.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff5aa85ec44db5ee"}},{"code_sha256_prefix":"076c252c52cbb161","entry":"split_list","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/model_caption_loader.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/model_caption_loader.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"076c252c52cbb161"}},{"code_sha256_prefix":"d6259b0d9750cac7","entry":"verify_response","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/model_mathvista.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/model_mathvista.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d6259b0d9750cac7"}},{"code_sha256_prefix":"c28eb86a8d14e4ed","entry":"get_most_similar","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/calculate_score.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/calculate_score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c28eb86a8d14e4ed"}},{"code_sha256_prefix":"e7a23ea3695f6437","entry":"safe_equal","repo":"luogen1996/llava-hr","repo_kind":"official","path":"llava_hr/eval/calculate_score.py","file_url":"https://github.com/luogen1996/llava-hr/blob/HEAD/llava_hr/eval/calculate_score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7a23ea3695f6437"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}