{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/modality-aware-neuron-pruning-for-unlearning","title":"Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models","arxiv_id":"2502.15910","date":"2025-02-21","proceeding":null,"authors":["Zheyuan Liu","Guangyao Dou","Xiangchi Yuan","Chunhui Zhang","Zhaoxuan Tan","Meng Jiang"],"abstract":"Generative models such as Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) trained on massive datasets can lead them to memorize and inadvertently reveal sensitive information, raising ethical and privacy concerns. While some prior works have explored this issue in the context of LLMs, it presents a unique challenge for MLLMs due to the entangled nature of knowledge across modalities, making comprehensive unlearning more difficult. To address this challenge, we propose Modality Aware Neuron Unlearning (MANU), a novel unlearning framework for MLLMs designed to selectively clip neurons based on their relative importance to the targeted forget data, curated for different modalities. Specifically, MANU consists of two stages: important neuron selection and selective pruning. The first stage identifies and collects the most influential neurons across modalities relative to the targeted forget knowledge, while the second stage is dedicated to pruning those selected neurons. MANU effectively isolates and removes the neurons that contribute most to the forget data within each modality, while preserving the integrity of retained knowledge. Our experiments conducted across various MLLM architectures illustrate that MANU can achieve a more balanced and comprehensive unlearning in each modality without largely affecting the overall model utility.","url_abs":"https://arxiv.org/abs/2502.15910v1","url_pdf":"https://arxiv.org/pdf/2502.15910v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"modality-aware-neuron-pruning-for-unlearning","repo_url":"https://github.com/franciscoliu/MANU","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.15910","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.15910"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/franciscoliu/MANU","reach":null}],"summary":{"ran_draft_wrong":6,"unverified":12},"by_repo_kind":{"official":{"samples":18,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":18,"samples":[{"code_sha256_prefix":"012682048242c85c","entry":"collect_feedforward_activations","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"012682048242c85c"}},{"code_sha256_prefix":"1936a6a620306829","entry":"compute_combined_scores_incremental","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1936a6a620306829"}},{"code_sha256_prefix":"884614df85440489","entry":"compute_top_k_pruning_mask","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"884614df85440489"}},{"code_sha256_prefix":"422f3314c273c0db","entry":"count_pruned_parameters","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"422f3314c273c0db"}},{"code_sha256_prefix":"b6ab11c5237058a8","entry":"train_collate_fn_idefics","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b6ab11c5237058a8"}},{"code_sha256_prefix":"9b5f99e80cb377b2","entry":"train_collate_fn_llava","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b5f99e80cb377b2"}},{"code_sha256_prefix":"01036113c76fb38d","entry":"ActivationCollector","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01036113c76fb38d"}},{"code_sha256_prefix":"c7b5cc3fa249a851","entry":"LLAVA_multimodal_Dataset","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7b5cc3fa249a851"}},{"code_sha256_prefix":"2812caca072f9365","entry":"LLAVA_unimodal_Dataset","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2812caca072f9365"}},{"code_sha256_prefix":"4786cdd3a43d1a0d","entry":"apply_mask_to_layer","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4786cdd3a43d1a0d"}},{"code_sha256_prefix":"04f7c6fa5a1e0db9","entry":"apply_structural_pruning","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04f7c6fa5a1e0db9"}},{"code_sha256_prefix":"b7aef12e8abb7da2","entry":"clear_all_memory","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7aef12e8abb7da2"}},{"code_sha256_prefix":"084a9497037ca179","entry":"get_most_available_gpu","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"084a9497037ca179"}},{"code_sha256_prefix":"ca29d3f8875120b9","entry":"iterative_pruning","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ca29d3f8875120b9"}},{"code_sha256_prefix":"195c0b426d03016d","entry":"load_model_and_processor","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"195c0b426d03016d"}},{"code_sha256_prefix":"9b08516f8e4fd5e2","entry":"print_memory_status","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b08516f8e4fd5e2"}},{"code_sha256_prefix":"975739fd25fbaca2","entry":"print_pruning_stats","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"975739fd25fbaca2"}},{"code_sha256_prefix":"0dea24d76b257587","entry":"register_feedforward_hooks","repo":"franciscoliu/MANU","repo_kind":"official","path":"prune/prune_incremental.py","file_url":"https://github.com/franciscoliu/MANU/blob/HEAD/prune/prune_incremental.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0dea24d76b257587"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}