{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-29716","title":"NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs","arxiv_id":"2605.29716","date":"2026-05-28","proceeding":"ICML","authors":["Shuaidi Wang","Zhan Zhuang","Ruping Huang","Yu Zhang"],"abstract":"Diffusion Large Language Models (dLLMs) have emerged as a promising non-autoregressive generative paradigm. Given the prohibitive computational cost of full fine-tuning, Parameter-Efficient Fine-Tuning (PEFT) has become the standard approach. However, existing PEFT methods (e.g., LoRA), originally tailored for autoregressive models, rely on static parameters that are agnostic to the noise level. Consequently, they ignore the intrinsic dynamics of the diffusion process, where input distributions and generation difficulty shift significantly along the denoising trajectory, rendering them suboptimal for dLLMs. To address this, we propose Noise-aware Low-Rank Adaptation (NaRA), which introduces a low-rank core matrix generated by a lightweight, globally shared hypernetwork conditioned on the noise level. This design enables the update matrices to vary continuously along the diffusion process while keeping parameter and latency overhead negligible. We provide a theoretical justification for the proposed NaRA framework and empirically demonstrate consistent improvements over noise-agnostic baselines across commonsense reasoning, mathematical reasoning, and code generation benchmarks. Our code is available at https://github.com/generaldi/NaRA.","url_abs":"https://arxiv.org/abs/2605.29716","url_pdf":"https://arxiv.org/pdf/2605.29716","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.29716","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.29716"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/generaldi/NaRA","reach":null}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ecf94b05123201e3","entry":"bloom_model_postprocess_past_key_value","repo":"generaldi/NaRA","repo_kind":"found_in_text","path":"nara/utils/other.py","file_url":"https://github.com/generaldi/NaRA/blob/HEAD/nara/utils/other.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecf94b05123201e3"}},{"code_sha256_prefix":"cbcc8d74cb56c97f","entry":"get_type","repo":"generaldi/NaRA","repo_kind":"found_in_text","path":"config/format_input.py","file_url":"https://github.com/generaldi/NaRA/blob/HEAD/config/format_input.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbcc8d74cb56c97f"}},{"code_sha256_prefix":"6cba5ffba64247d1","entry":"prepare_model_for_int8_training","repo":"generaldi/NaRA","repo_kind":"found_in_text","path":"nara/utils/other.py","file_url":"https://github.com/generaldi/NaRA/blob/HEAD/nara/utils/other.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cba5ffba64247d1"}},{"code_sha256_prefix":"5af2b785cc55d3ea","entry":"shift_tokens_right","repo":"generaldi/NaRA","repo_kind":"found_in_text","path":"nara/utils/other.py","file_url":"https://github.com/generaldi/NaRA/blob/HEAD/nara/utils/other.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5af2b785cc55d3ea"}},{"code_sha256_prefix":"7e416ebc9854a9ee","entry":"compute_original_dream_loss","repo":"generaldi/NaRA","repo_kind":"found_in_text","path":"utils/loss.py","file_url":"https://github.com/generaldi/NaRA/blob/HEAD/utils/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e416ebc9854a9ee"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.AI","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}