{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-06022","title":"AdaFuse: Adaptive Ensemble Decoding with Test-Time Scaling for LLMs","arxiv_id":"2601.06022","date":"2026-01-09","proceeding":null,"authors":["Chengming Cui","Tianxin Wei","Ziyi Chen","Ruizhong Qiu","Zhichen Zeng","Zhining Liu","Xuying Ning","Duo Zhou","Jingrui He"],"abstract":"Large language models (LLMs) exhibit complementary strengths arising from differences in pretraining data, model architectures, and decoding behaviors. Inference-time ensembling provides a practical way to combine these capabilities without retraining. However, existing ensemble approaches suffer from fundamental limitations. Most rely on fixed fusion granularity, which lacks the flexibility required for mid-generation adaptation and fails to adapt to different generation characteristics across tasks. To address these challenges, we propose AdaFuse, an adaptive ensemble decoding framework that dynamically selects semantically appropriate fusion units during generation. Rather than committing to a fixed granularity, AdaFuse adjusts fusion behavior on the fly based on the decoding context, with words serving as basic building blocks for alignment. To be specific, we introduce an uncertainty-based criterion to decide whether to apply ensembling at each decoding step. Under confident decoding states, the model continues generation directly. In less certain states, AdaFuse invokes a diversity-aware scaling strategy to explore alternative candidate continuations and inform ensemble decisions. This design establishes a synergistic interaction between adaptive ensembling and test-time scaling, where ensemble decisions guide targeted exploration, and the resulting diversity in turn strengthens ensemble quality. Experiments on open-domain question answering, arithmetic reasoning, and machine translation demonstrate that AdaFuse consistently outperforms strong ensemble baselines, achieving an average relative improvement of 6.88%. The code is available at https://github.com/CCM0111/AdaFuse.","url_abs":"https://arxiv.org/abs/2601.06022","url_pdf":"https://arxiv.org/pdf/2601.06022","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.06022","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.06022"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/CCM0111/AdaFuse","reach":null}],"summary":{"ran_draft_wrong":5,"ran_honours":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"cb85933171775187","entry":"clean_text_before_question","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb85933171775187"}},{"code_sha256_prefix":"228f537a3be632db","entry":"compute_top1_margin","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"228f537a3be632db"}},{"code_sha256_prefix":"4522512ab06aaf5a","entry":"create_gsm_collate_fn","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4522512ab06aaf5a"}},{"code_sha256_prefix":"c0319de0754bc4c0","entry":"create_qa_collate_fn","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0319de0754bc4c0"}},{"code_sha256_prefix":"40e202a1ceb43583","entry":"extract_boxed_answer","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40e202a1ceb43583"}},{"code_sha256_prefix":"cb34d48e266a5120","entry":"extract_target_question","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb34d48e266a5120"}},{"code_sha256_prefix":"3c0cb4a41dbe1960","entry":"extract_word_tokens_from_generated","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c0cb4a41dbe1960"}},{"code_sha256_prefix":"4b3861004b50a529","entry":"compute_span_perplexity","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b3861004b50a529"}},{"code_sha256_prefix":"1ad23c1a4fe48e22","entry":"ensemble_decoding_worker","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ad23c1a4fe48e22"}},{"code_sha256_prefix":"018ca54f3f1d21af","entry":"generate_candidate_spans","repo":"CCM0111/AdaFuse","repo_kind":"found_in_text","path":"AdaFuse_two_models.py","file_url":"https://github.com/CCM0111/AdaFuse/blob/HEAD/AdaFuse_two_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"018ca54f3f1d21af"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}