{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/noesis-differentially-private-knowledge","title":"NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation","arxiv_id":"2504.18147","date":"2025-04-25","proceeding":null,"authors":["Rob Romijnders","Stefanos Laskaridis","Ali Shahin Shamsabadi","Hamed Haddadi"],"abstract":"Large Language Models (LLM) are typically trained on vast amounts of data from various sources. Even when designed modularly (e.g., Mixture-of-Experts), LLMs can leak privacy on their sources. Conversely, training such models in isolation arguably prohibits generalization. To this end, we propose a framework, NoEsis, which builds upon the desired properties of modularity, privacy, and knowledge transfer. NoEsis integrates differential privacy with a hybrid two-staged parameter-efficient fine-tuning that combines domain-specific low-rank adapters, acting as experts, with common prompt tokens, acting as a knowledge-sharing backbone. Results from our evaluation on CodeXGLUE showcase that NoEsis can achieve provable privacy guarantees with tangible knowledge transfer across domains, and empirically show protection against Membership Inference Attacks. Finally, on code completion tasks, NoEsis bridges at least 77% of the accuracy gap between the non-shared and the non-private baseline.","url_abs":"https://arxiv.org/abs/2504.18147v1","url_pdf":"https://arxiv.org/pdf/2504.18147v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"code-completion","task_name":"Code Completion"},{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2504.18147","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.18147"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/RobRomijnders/noesis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"found_in_text":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b34ef4a96f55b090","entry":"custom_collate_fn","repo":"RobRomijnders/noesis","repo_kind":"found_in_text","path":"NoEsis/util_data.py","file_url":"https://github.com/RobRomijnders/noesis/blob/HEAD/NoEsis/util_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b34ef4a96f55b090"}},{"code_sha256_prefix":"0a3b80da511388ad","entry":"get_auroc","repo":"RobRomijnders/noesis","repo_kind":"found_in_text","path":"NoEsis/util_mia.py","file_url":"https://github.com/RobRomijnders/noesis/blob/HEAD/NoEsis/util_mia.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a3b80da511388ad"}},{"code_sha256_prefix":"ca143f55f88b9020","entry":"has_multiple_datasets","repo":"RobRomijnders/noesis","repo_kind":"found_in_text","path":"NoEsis/util.py","file_url":"https://github.com/RobRomijnders/noesis/blob/HEAD/NoEsis/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca143f55f88b9020"}},{"code_sha256_prefix":"3115c9efb8a27354","entry":"read_config","repo":"RobRomijnders/noesis","repo_kind":"found_in_text","path":"NoEsis/util_model.py","file_url":"https://github.com/RobRomijnders/noesis/blob/HEAD/NoEsis/util_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3115c9efb8a27354"}},{"code_sha256_prefix":"088eab3c4c6b5808","entry":"strip_datasets","repo":"RobRomijnders/noesis","repo_kind":"found_in_text","path":"NoEsis/util.py","file_url":"https://github.com/RobRomijnders/noesis/blob/HEAD/NoEsis/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"088eab3c4c6b5808"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}