{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fast-best-of-n-decoding-via-speculative","title":"Fast Best-of-N Decoding via Speculative Rejection","arxiv_id":"2410.20290","date":"2024-10-26","proceeding":null,"authors":["Hanshi Sun","Momin Haider","Ruiqi Zhang","Huitao Yang","Jiahao Qiu","Ming Yin","Mengdi Wang","Peter Bartlett","Andrea Zanette"],"abstract":"The safe and effective deployment of Large Language Models (LLMs) involves a critical step called alignment, which ensures that the model's responses are in accordance with human preferences. Prevalent alignment techniques, such as DPO, PPO and their variants, align LLMs by changing the pre-trained model weights during a phase called post-training. While predominant, these post-training methods add substantial complexity before LLMs can be deployed. Inference-time alignment methods avoid the complex post-training step and instead bias the generation towards responses that are aligned with human preferences. The best-known inference-time alignment method, called Best-of-N, is as effective as the state-of-the-art post-training procedures. Unfortunately, Best-of-N requires vastly more resources at inference time than standard decoding strategies, which makes it computationally not viable. In this work, we introduce Speculative Rejection, a computationally-viable inference-time alignment algorithm. It generates high-scoring responses according to a given reward model, like Best-of-N does, while being between 16 to 32 times more computationally efficient.","url_abs":"https://arxiv.org/abs/2410.20290v2","url_pdf":"https://arxiv.org/pdf/2410.20290v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fast-best-of-n-decoding-via-speculative","repo_url":"https://github.com/Zanette-Labs/SpeculativeRejection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"dpo","method_name":"DPO"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"ppo","method_name":"PPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.20290","atlas_url":"https://app.syntology.ai/?focus=2410.20290","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.20290"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Zanette-Labs/SpeculativeRejection","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"72df4dc495e2d85d","entry":"get_alpha_value","repo":"Zanette-Labs/SpeculativeRejection","repo_kind":"official","path":"postprocess/plot_compare.py","file_url":"https://github.com/Zanette-Labs/SpeculativeRejection/blob/HEAD/postprocess/plot_compare.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72df4dc495e2d85d"}},{"code_sha256_prefix":"9d7befac84db2dac","entry":"get_json_filepaths","repo":"Zanette-Labs/SpeculativeRejection","repo_kind":"official","path":"postprocess/plot_compare.py","file_url":"https://github.com/Zanette-Labs/SpeculativeRejection/blob/HEAD/postprocess/plot_compare.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d7befac84db2dac"}},{"code_sha256_prefix":"5d0741c5a48780fc","entry":"get_num_gpus","repo":"Zanette-Labs/SpeculativeRejection","repo_kind":"official","path":"postprocess/plot_compare.py","file_url":"https://github.com/Zanette-Labs/SpeculativeRejection/blob/HEAD/postprocess/plot_compare.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d0741c5a48780fc"}},{"code_sha256_prefix":"a8707a4b3d8b3abd","entry":"get_parsed_data","repo":"Zanette-Labs/SpeculativeRejection","repo_kind":"official","path":"postprocess/gather_best_ans.py","file_url":"https://github.com/Zanette-Labs/SpeculativeRejection/blob/HEAD/postprocess/gather_best_ans.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8707a4b3d8b3abd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}