{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-consistent-decoding-for-more-factual","title":"Self-Consistent Decoding for More Factual Open Responses","arxiv_id":"2403.00696","date":"2024-03-01","proceeding":null,"authors":["Christopher Malon","Xiaodan Zhu"],"abstract":"Self-consistency has emerged as a powerful method for improving the accuracy of short answers generated by large language models. As previously defined, it only concerns the accuracy of a final answer parsed from generated text. In this work, we extend the idea to open response generation, by integrating voting into the decoding method. Each output sentence is selected from among multiple samples, conditioning on the previous selections, based on a simple token overlap score. We compare this \"Sample & Select\" method to greedy decoding, beam search, nucleus sampling, and the recently introduced hallucination avoiding decoders of DoLA, P-CRR, and S-CRR. We show that Sample & Select improves factuality by a 30% relative margin against these decoders in NLI-based evaluation on the subsets of CNN/DM and XSum used in the FRANK benchmark, while maintaining comparable ROUGE-1 F1 scores against reference summaries. We collect human verifications of the generated summaries, confirming the factual superiority of our method.","url_abs":"https://arxiv.org/abs/2403.00696v1","url_pdf":"https://arxiv.org/pdf/2403.00696v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-consistent-decoding-for-more-factual","repo_url":"https://github.com/cdmalon/selfconsistent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"response-generation","task_name":"Response Generation"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.00696","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.00696"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cdmalon/selfconsistent","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cd1321a0fb9b3888","entry":"wrap_error","repo":"cdmalon/selfconsistent","repo_kind":"official","path":"src/llama2/flask_2048_beam_13b.py","file_url":"https://github.com/cdmalon/selfconsistent/blob/HEAD/src/llama2/flask_2048_beam_13b.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd1321a0fb9b3888"}},{"code_sha256_prefix":"62f475930d56912f","entry":"wrap_message","repo":"cdmalon/selfconsistent","repo_kind":"official","path":"src/llama2/flask_2048_beam_13b.py","file_url":"https://github.com/cdmalon/selfconsistent/blob/HEAD/src/llama2/flask_2048_beam_13b.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62f475930d56912f"}},{"code_sha256_prefix":"b83869a5b78f7e51","entry":"wrap_predictions","repo":"cdmalon/selfconsistent","repo_kind":"official","path":"src/llama2/flask_2048_beam_13b.py","file_url":"https://github.com/cdmalon/selfconsistent/blob/HEAD/src/llama2/flask_2048_beam_13b.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b83869a5b78f7e51"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}