{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-01347","title":"MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate","arxiv_id":"2605.01347","date":"2026-05-02","proceeding":null,"authors":["Jianze Wang","Ying Liu","Jinlong Chen","Xuchun Hu","Qilong Zhang","Yu Cao","Jun Wang","Hua Yang","Yong Xie","Qianglong Chen"],"abstract":"On-policy distillation (OPD) trains a student on its own trajectories under token-level teacher supervision, but existing methods are capped by a single-teacher capability ceiling: when the teacher errs, the student inherits the error. OPD also remains largely unexplored in agentic tasks, where per-step errors compound across long trajectories and destabilize training. We propose MAD-OPD (Multi-Agent Debate-driven On-Policy Distillation), which breaks this ceiling by recasting the distillation teacher as a deliberative collective of teachers that debate over the student's on-policy state; the debate produces an emergent collective intelligence that supplies token-level supervision, with each teacher's contribution weighted by its post-debate confidence. To extend OPD to agentic tasks, we also introduce On-Policy Agentic Distillation (OPAD), which adds step-level sampling to stabilize training under multi-step error compounding. We additionally derive a task-adaptive divergence principle, selecting JSD (Jensen-Shannon divergence) for agentic stability and reverse KL (Kullback-Leibler) divergence for code generation, and verify it both theoretically and empirically. Across six teacher-student configurations (Qwen3 and Qwen3.5; 1.7B-14B students, 8B-32B teachers) and five agentic and code benchmarks, MAD-OPD ranks first across all six configurations; on the 14B+8B$\\to$4B setting it lifts the agentic average by $+2.4\\%$ and the code average by $+3.7\\%$ over the stronger single-teacher OPD.","url_abs":"https://arxiv.org/abs/2605.01347","url_pdf":"https://arxiv.org/pdf/2605.01347","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.01347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.01347"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/sierra-research/tau2-bench","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/chiefovoavicii/MAD-OPD","reach":null}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":7,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0e1894243144defb","entry":"audio_bytes_to_string","repo":"sierra-research/tau2-bench","repo_kind":"found_in_text","path":"src/tau2/data_model/audio.py","file_url":"https://github.com/sierra-research/tau2-bench/blob/HEAD/src/tau2/data_model/audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e1894243144defb"}},{"code_sha256_prefix":"9bc9c67906873029","entry":"audio_string_to_bytes","repo":"sierra-research/tau2-bench","repo_kind":"found_in_text","path":"src/tau2/data_model/audio.py","file_url":"https://github.com/sierra-research/tau2-bench/blob/HEAD/src/tau2/data_model/audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9bc9c67906873029"}},{"code_sha256_prefix":"c754b06031b4c93b","entry":"generalized_jsd_loss_single","repo":"chiefovoavicii/MAD-OPD","repo_kind":"found_in_text","path":"mad_opd/trainers/mad_opd_core.py","file_url":"https://github.com/chiefovoavicii/MAD-OPD/blob/HEAD/mad_opd/trainers/mad_opd_core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c754b06031b4c93b"}},{"code_sha256_prefix":"ff018fdd1576b054","entry":"run","repo":"chiefovoavicii/MAD-OPD","repo_kind":"found_in_text","path":"eval/mbpp_plus_eval.py","file_url":"https://github.com/chiefovoavicii/MAD-OPD/blob/HEAD/eval/mbpp_plus_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff018fdd1576b054"}},{"code_sha256_prefix":"97e7067f2d857daf","entry":"wait_for_vllm","repo":"chiefovoavicii/MAD-OPD","repo_kind":"found_in_text","path":"eval/bfcl_eval.py","file_url":"https://github.com/chiefovoavicii/MAD-OPD/blob/HEAD/eval/bfcl_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"97e7067f2d857daf"}},{"code_sha256_prefix":"ca7a3c7e38226c29","entry":"multi_teacher_jsd_loss","repo":"chiefovoavicii/MAD-OPD","repo_kind":"found_in_text","path":"mad_opd/trainers/mad_opd_core.py","file_url":"https://github.com/chiefovoavicii/MAD-OPD/blob/HEAD/mad_opd/trainers/mad_opd_core.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ca7a3c7e38226c29"}},{"code_sha256_prefix":"0e1c52741a32644c","entry":"wait_for_vllm","repo":"chiefovoavicii/MAD-OPD","repo_kind":"found_in_text","path":"eval/tau2_bench_eval.py","file_url":"https://github.com/chiefovoavicii/MAD-OPD/blob/HEAD/eval/tau2_bench_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e1c52741a32644c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}