{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/intent-factored-generation-unleashing-the","title":"Intent Factored Generation: Unleashing the Diversity in Your Language Model","arxiv_id":"2506.09659","date":"2025-06-11","proceeding":null,"authors":["Eltayeb Ahmed","Uljad Berdica","Martha Elliott","Danijela Horak","Jakob N. Foerster"],"abstract":"Obtaining multiple meaningfully diverse, high quality samples from Large Language Models for a fixed prompt remains an open challenge. Current methods for increasing diversity often only operate at the token-level, paraphrasing the same response. This is problematic because it leads to poor exploration on reasoning problems and to unengaging, repetitive conversational agents. To address this we propose Intent Factored Generation (IFG), factorising the sampling process into two stages. First, we sample a semantically dense intent, e.g., a summary or keywords. Second, we sample the final response conditioning on both the original prompt and the intent from the first stage. This allows us to use a higher temperature during the intent step to promote conceptual diversity, and a lower temperature during the final generation to ensure the outputs are coherent and self-consistent. Additionally, we find that prompting the model to explicitly state its intent for each step of the chain-of-thought before generating the step is beneficial for reasoning tasks. We demonstrate our method's effectiveness across a diverse set of tasks. We show this method improves both pass@k and Reinforcement Learning from Verifier Feedback on maths and code tasks. For instruction-tuning, we combine IFG with Direct Preference Optimisation to increase conversational diversity without sacrificing reward. Finally, we achieve higher diversity while maintaining the quality of generations on a general language modelling task, using a new dataset of reader comments and news articles that we collect and open-source. In summary, we present a simple method of increasing the sample diversity of LLMs while maintaining performance. This method can be implemented by changing the prompt and varying the temperature during generation, making it easy to integrate into many algorithms for gains across various applications.","url_abs":"https://arxiv.org/abs/2506.09659v1","url_pdf":"https://arxiv.org/pdf/2506.09659v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"intent-factored-generation-unleashing-the","repo_url":"https://github.com/flairox/ifg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"articles","task_name":"Articles"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2506.09659","atlas_url":"https://app.syntology.ai/?focus=2506.09659","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.09659"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/flairox/ifg","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e6b10b3a46a1e73c","entry":"compare_final_answer_to_ground_truth","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/math_eval.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/math_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6b10b3a46a1e73c"}},{"code_sha256_prefix":"25929c22eda9938b","entry":"compute_performance_metrics","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/ifg_infer_and_score.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/ifg_infer_and_score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25929c22eda9938b"}},{"code_sha256_prefix":"b881ac314109956a","entry":"create_dataset_of_size_batch_size","repo":"flairox/ifg","repo_kind":"official","path":"training/tune_batch_size.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/training/tune_batch_size.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b881ac314109956a"}},{"code_sha256_prefix":"e213f105ef5bfbfa","entry":"evaluate_comments","repo":"flairox/ifg","repo_kind":"official","path":"unstructured_tasks/metrics/reward_model_eval.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/unstructured_tasks/metrics/reward_model_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e213f105ef5bfbfa"}},{"code_sha256_prefix":"606bde9343246a50","entry":"format_duration","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/experiment_pipelines/hparam_sweeper.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/experiment_pipelines/hparam_sweeper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"606bde9343246a50"}},{"code_sha256_prefix":"79035c5cd6888332","entry":"generate_subpaths","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/star.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/star.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"79035c5cd6888332"}},{"code_sha256_prefix":"b99c690a18bac742","entry":"get_bound","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/experiment_pipelines/hparam_sweeper.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/experiment_pipelines/hparam_sweeper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b99c690a18bac742"}},{"code_sha256_prefix":"adc1130833a4f725","entry":"get_result","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/experiment_pipelines/hparam_sweeper.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/experiment_pipelines/hparam_sweeper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"adc1130833a4f725"}},{"code_sha256_prefix":"e15df122cc85e5a6","entry":"preprocess_sample","repo":"flairox/ifg","repo_kind":"official","path":"training/finetune_completion.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/training/finetune_completion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e15df122cc85e5a6"}},{"code_sha256_prefix":"1ab57d52c0087750","entry":"process_docs","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/math_eval.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/math_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ab57d52c0087750"}},{"code_sha256_prefix":"90f8729aba04c979","entry":"process_results","repo":"flairox/ifg","repo_kind":"official","path":"hendrycks_math/math_eval.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/hendrycks_math/math_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90f8729aba04c979"}},{"code_sha256_prefix":"196f3f69c8d074dd","entry":"read_prompt_file","repo":"flairox/ifg","repo_kind":"official","path":"training/finetune_completion.py","file_url":"https://github.com/flairox/ifg/blob/HEAD/training/finetune_completion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"196f3f69c8d074dd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}