{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lvlm-intrepret-an-interpretability-tool-for","title":"LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models","arxiv_id":"2404.03118","date":"2024-04-03","proceeding":null,"authors":["Gabriela Ben Melech Stan","Estelle Aflalo","Raanan Yehezkel Rohekar","Anahita Bhiwandiwalla","Shao-Yen Tseng","Matthew Lyle Olson","Yaniv Gurwicz","Chenfei Wu","Nan Duan","Vasudev Lal"],"abstract":"In the rapidly evolving landscape of artificial intelligence, multi-modal large language models are emerging as a significant area of interest. These models, which combine various forms of data input, are becoming increasingly popular. However, understanding their internal mechanisms remains a complex task. Numerous advancements have been made in the field of explainability tools and mechanisms, yet there is still much to explore. In this work, we present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer, and assess the efficacy of the language model in grounding its output in the image. With our application, a user can systematically investigate the model and uncover system limitations, paving the way for enhancements in system capabilities. Finally, we present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.","url_abs":"https://arxiv.org/abs/2404.03118v3","url_pdf":"https://arxiv.org/pdf/2404.03118v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lvlm-intrepret-an-interpretability-tool-for","repo_url":"https://github.com/IntelLabs/lvlm-interpret","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.03118","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.03118"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IntelLabs/lvlm-interpret","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":10,"unverified":2},"by_repo_kind":{"official":{"samples":12,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"27481013d8c59da1","entry":"avg_heads","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_relevancy.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_relevancy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"27481013d8c59da1"}},{"code_sha256_prefix":"23f50ad92e876f94","entry":"convert_token2word","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_attn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_attn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23f50ad92e876f94"}},{"code_sha256_prefix":"248b40944c6c5f7a","entry":"draw_heatmap_on_image","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_attn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_attn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"248b40944c6c5f7a"}},{"code_sha256_prefix":"b5f865fcf0203ef4","entry":"get_expla_set_per_rad","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_causal_discovery_fn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_causal_discovery_fn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b5f865fcf0203ef4"}},{"code_sha256_prefix":"37fdd8a0f972bd45","entry":"get_relevant_image_tokens","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_causal_discovery_fn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_causal_discovery_fn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37fdd8a0f972bd45"}},{"code_sha256_prefix":"35b3b196a6b44a41","entry":"get_relevant_prompt_tokens","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_causal_discovery_fn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_causal_discovery_fn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35b3b196a6b44a41"}},{"code_sha256_prefix":"0f9f25db5c4ac426","entry":"handle_self_attention_image_vit","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_relevancy.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_relevancy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f9f25db5c4ac426"}},{"code_sha256_prefix":"309bc258636e0fd3","entry":"move_to_device","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_attn.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_attn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"309bc258636e0fd3"}},{"code_sha256_prefix":"6db878fefb797d88","entry":"process_image","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6db878fefb797d88"}},{"code_sha256_prefix":"57154e2c77c27b6b","entry":"to_gradio_chatbot","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"57154e2c77c27b6b"}},{"code_sha256_prefix":"d81ca3a9ee15e007","entry":"get_processor_model","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d81ca3a9ee15e007"}},{"code_sha256_prefix":"87783f89552e57bf","entry":"handle_self_attention_image","repo":"IntelLabs/lvlm-interpret","repo_kind":"official","path":"utils_relevancy.py","file_url":"https://github.com/IntelLabs/lvlm-interpret/blob/HEAD/utils_relevancy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"87783f89552e57bf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}