{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-09536","title":"TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM","arxiv_id":"2605.09536","date":"2026-05-10","proceeding":null,"authors":["Haoyang Zhou","Li Kong","Shijie Ren","Xiting Wang","Shuang Liang","Guowei Wang","Zhenxuan Pan"],"abstract":"Diffusion large language models (dLLMs) offer a promising paradigm for parallel text generation, but in practice they face an accuracy-parallelism trade-off, where increasing tokens per forward (TPF) often degrades generation quality. Existing acceleration methods often gain speed at the cost of accuracy. To address this limitation, we propose TAD, a Temporal-Aware trajectory self-Distillation framework. During data construction, we condition a teacher model on both the prompt and the ground-truth response to generate decoding trajectories, recording the intermediate masked states throughout the process. Based on how many decoding steps remain before each masked token is revealed, we partition masked positions into near and distant subsets. For near tokens, we train the student with a hard cross-entropy loss using the teacher trajectory tokens as labels, encouraging confident predictions for tokens that are about to be decoded. For distant tokens, we apply a soft KL divergence loss between the teacher and student token distributions, providing softer supervision and preserving future planning knowledge. This temporal-aware partition naturally gives rise to two deployment configurations: a Quality model that prioritizes accuracy and a Speed model that favors more aggressive acceleration. Experiments show that TAD consistently improves the accuracy-parallelism trade-off. On LLaDA, it raises average accuracy from 46.2\\% to 51.6\\% with the Quality model and average AUP from 46.2 to 257.1 with the Speed model. Our code is available at: https://github.com/BHmingyang/TAD","url_abs":"https://arxiv.org/abs/2605.09536","url_pdf":"https://arxiv.org/pdf/2605.09536","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.09536","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.09536"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/BHmingyang/TAD","reach":null}],"summary":{"ran":8,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"027b7c000eacca21","entry":"add_gumbel_noise","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"eval/eval_llada.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/eval/eval_llada.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"027b7c000eacca21"}},{"code_sha256_prefix":"ccf7022c754f65a6","entry":"get_aup","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"AUP_leaderboard/aup_utils.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/AUP_leaderboard/aup_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ccf7022c754f65a6"}},{"code_sha256_prefix":"af22e74b922388df","entry":"get_deepspeed_config","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"Dream/train_dream.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/Dream/train_dream.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af22e74b922388df"}},{"code_sha256_prefix":"befea8ff69c3c190","entry":"get_num_transfer_tokens","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"eval/eval_llada.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/eval/eval_llada.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"befea8ff69c3c190"}},{"code_sha256_prefix":"cbd81f168febc18c","entry":"get_transfer_index_entropy","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"eval/eval_llada.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/eval/eval_llada.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cbd81f168febc18c"}},{"code_sha256_prefix":"916e2a53aa89c469","entry":"load_config","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"Dream/merge_lora.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/Dream/merge_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"916e2a53aa89c469"}},{"code_sha256_prefix":"115589cfba00cf2c","entry":"load_config","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"Dream/train_dream.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/Dream/train_dream.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"115589cfba00cf2c"}},{"code_sha256_prefix":"b117946509018b9a","entry":"weight_function","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"AUP_leaderboard/aup_utils.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/AUP_leaderboard/aup_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b117946509018b9a"}},{"code_sha256_prefix":"419b9da5c8cf99a3","entry":"generate","repo":"BHmingyang/TAD","repo_kind":"found_in_text","path":"eval/generate.py","file_url":"https://github.com/BHmingyang/TAD/blob/HEAD/eval/generate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"419b9da5c8cf99a3"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}