{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-15346","title":"When to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensembling","arxiv_id":"2510.15346","date":"2025-10-17","proceeding":"ICLR","authors":["Heecheol Yun","Kwangmin Ki","Junghyun Lee","Eunho Yang"],"abstract":"Ensembling Large Language Models (LLMs) has gained attention as a promising approach to surpass the performance of individual models by leveraging their complementary strengths. In particular, aggregating models' next-token probability distributions to select the next token has been shown to be effective in various tasks. However, while successful for short-form answers, its application to long-form generation remains underexplored. In this paper, we show that using existing ensemble methods in long-form generation requires a careful choice of ensembling positions, since the standard practice of ensembling at every token often degrades performance. We identify two key factors for determining the ensembling positions: tokenization mismatch across models and consensus in their next-token probability distributions. Based on this, we propose SAFE, (Stable And Fast LLM Ensembling), a framework that selectively ensembles by jointly considering these factors. To further improve stability, we apply a probability sharpening strategy when the ensemble distribution becomes overly smooth, enabling the selection of more confident tokens during ensembling. Our experiments on diverse benchmarks, including MATH500 and BBH, demonstrate that SAFE outperforms existing methods in both accuracy and efficiency, with gains achieved even when ensembling fewer than 1% of tokens.","url_abs":"https://arxiv.org/abs/2510.15346","url_pdf":"https://arxiv.org/pdf/2510.15346","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2510.15346","atlas_url":"https://app.syntology.ai/?focus=2510.15346","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.15346"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/yoon6503/SAFE","reach":{"status":"ok"}}],"summary":{"unverified":14},"by_repo_kind":{"found_in_text":{"samples":14,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"8f6d8e2c52477e99","entry":"check_byte_mappings","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_gac.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_gac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f6d8e2c52477e99"}},{"code_sha256_prefix":"c8947f39a6f3b186","entry":"format_example","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"run_safe.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/run_safe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8947f39a6f3b186"}},{"code_sha256_prefix":"914abaca965ea62f","entry":"format_example","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"run_safe_mmlu.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/run_safe_mmlu.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"914abaca965ea62f"}},{"code_sha256_prefix":"af06fa7b0799844f","entry":"generate_single","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"generate_utils.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/generate_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af06fa7b0799844f"}},{"code_sha256_prefix":"5e6a0610bd64a373","entry":"get_top_k_tokens","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_unite.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_unite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e6a0610bd64a373"}},{"code_sha256_prefix":"46d9414b894b9482","entry":"get_union_vocab","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_unite.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_unite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"46d9414b894b9482"}},{"code_sha256_prefix":"54de62c71b112f59","entry":"get_union_vocab","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_unite_three.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_unite_three.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54de62c71b112f59"}},{"code_sha256_prefix":"73bded896c5b3ffa","entry":"load_models_tokenizer","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"run_safe.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/run_safe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73bded896c5b3ffa"}},{"code_sha256_prefix":"eb44ad2ef1751c0b","entry":"prune_cache","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"caching.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/caching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb44ad2ef1751c0b"}},{"code_sha256_prefix":"2e507b94b63c62d4","entry":"prune_dynamic_cache","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"caching.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/caching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2e507b94b63c62d4"}},{"code_sha256_prefix":"13d0e4b4a4a6bae9","entry":"prune_tuple_cache","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"caching.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/caching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13d0e4b4a4a6bae9"}},{"code_sha256_prefix":"e008364719672578","entry":"setup_custom_logger","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"logger.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/logger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e008364719672578"}},{"code_sha256_prefix":"67d7a56dc464538f","entry":"update_vocab","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_unite.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_unite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67d7a56dc464538f"}},{"code_sha256_prefix":"d225287774b5b272","entry":"update_vocab","repo":"yoon6503/SAFE","repo_kind":"found_in_text","path":"safe_generate_unite_three.py","file_url":"https://github.com/yoon6503/SAFE/blob/HEAD/safe_generate_unite_three.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d225287774b5b272"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}