{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-multi-modal-models-can-interpret","title":"Large Multi-modal Models Can Interpret Features in Large Multi-modal Models","arxiv_id":"2411.14982","date":"2024-11-22","proceeding":null,"authors":["Kaichen Zhang","Yifei Shen","Bo Li","Ziwei Liu"],"abstract":"Recent advances in Large Multimodal Models (LMMs) lead to significant breakthroughs in both academia and industry. One question that arises is how we, as humans, can understand their internal neural representations. This paper takes an initial step towards addressing this question by presenting a versatile framework to identify and interpret the semantics within LMMs. Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the representations into human understandable features. 2) We then present an automatic interpretation framework to interpreted the open-semantic features learned in SAE by the LMMs themselves. We employ this framework to analyze the LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these features can effectively steer the model's behavior. Our results contribute to a deeper understanding of why LMMs excel in specific tasks, including EQ tests, and illuminate the nature of their mistakes along with potential strategies for their rectification. These findings offer new insights into the internal mechanisms of LMMs and suggest parallels with the cognitive processes of the human brain.","url_abs":"https://arxiv.org/abs/2411.14982v1","url_pdf":"https://arxiv.org/pdf/2411.14982v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-multi-modal-models-can-interpret","repo_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.14982","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.14982"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"5bb83dccc29d1248","entry":"create_response_model","repo":"EvolvingLMMs-Lab/multimodal-sae","repo_kind":"official","path":"sae_auto_interp/clients/client.py","file_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae/blob/HEAD/sae_auto_interp/clients/client.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5bb83dccc29d1248"}},{"code_sha256_prefix":"d2a23ff3b12659f5","entry":"load_explanation","repo":"EvolvingLMMs-Lab/multimodal-sae","repo_kind":"official","path":"sae_auto_interp/utils.py","file_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae/blob/HEAD/sae_auto_interp/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d2a23ff3b12659f5"}},{"code_sha256_prefix":"4a2d95aa70e89aa9","entry":"load_filter","repo":"EvolvingLMMs-Lab/multimodal-sae","repo_kind":"official","path":"sae_auto_interp/utils.py","file_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae/blob/HEAD/sae_auto_interp/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4a2d95aa70e89aa9"}},{"code_sha256_prefix":"3f189915182fa1cf","entry":"load_tokenized_data","repo":"EvolvingLMMs-Lab/multimodal-sae","repo_kind":"official","path":"sae_auto_interp/utils.py","file_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae/blob/HEAD/sae_auto_interp/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3f189915182fa1cf"}},{"code_sha256_prefix":"bbb7be8ec437c457","entry":"process_wrapper","repo":"EvolvingLMMs-Lab/multimodal-sae","repo_kind":"official","path":"sae_auto_interp/pipeline.py","file_url":"https://github.com/EvolvingLMMs-Lab/multimodal-sae/blob/HEAD/sae_auto_interp/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bbb7be8ec437c457"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}