{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-reason-for-long-form-story","title":"Learning to Reason for Long-Form Story Generation","arxiv_id":"2503.22828","date":"2025-03-28","proceeding":null,"authors":["Alexander Gurung","Mirella Lapata"],"abstract":"Generating high-quality stories spanning thousands of tokens requires competency across a variety of skills, from tracking plot and character arcs to keeping a consistent and engaging style. Due to the difficulty of sourcing labeled datasets and precise quality measurements, most work using large language models (LLMs) for long-form story generation uses combinations of hand-designed prompting techniques to elicit author-like behavior. This is a manual process that is highly dependent on the specific story-generation task. Motivated by the recent success of applying RL with Verifiable Rewards to domains like math and coding, we propose a general story-generation task (Next-Chapter Prediction) and a reward formulation (Verified Rewards via Completion Likelihood Improvement) that allows us to use an unlabeled book dataset as a learning signal for reasoning. We learn to reason over a story's condensed information and generate a detailed plan for the next chapter. Our reasoning is evaluated via the chapters it helps a story-generator create, and compared against non-trained and supervised finetuning (SFT) baselines. Pairwise human judgments reveal the chapters our learned reasoning produces are preferred across almost all metrics, and the effect is more pronounced in Scifi and Fantasy genres.","url_abs":"https://arxiv.org/abs/2503.22828v1","url_pdf":"https://arxiv.org/pdf/2503.22828v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-reason-for-long-form-story","repo_url":"https://github.com/Alex-Gurung/ReasoningNCP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"form","task_name":"Form"},{"task_slug":"math","task_name":"Math"},{"task_slug":"story-generation","task_name":"Story Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.22828","atlas_url":"https://app.syntology.ai/?focus=2503.22828","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.22828"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Alex-Gurung/ReasoningNCP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":14,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":15,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eafb4e8880093be5","entry":"compile_prompt_components","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/prompt_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/prompt_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eafb4e8880093be5"}},{"code_sha256_prefix":"d1241773b3d36cda","entry":"find_index_of_last_system_message","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/compute_baseline_ppls.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/compute_baseline_ppls.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1241773b3d36cda"}},{"code_sha256_prefix":"a788a362d50886d9","entry":"find_index_of_last_system_message","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/ray_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/ray_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a788a362d50886d9"}},{"code_sha256_prefix":"890e43429501c9fa","entry":"format_questions_and_answer_messages","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/chiron_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/chiron_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"890e43429501c9fa"}},{"code_sha256_prefix":"4503ae73cbd5819b","entry":"format_snippet_for_chiron_generation","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/chiron_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/chiron_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4503ae73cbd5819b"}},{"code_sha256_prefix":"13d0ee715209614e","entry":"get_chiron_generation_question_text","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/chiron_generation_module_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/chiron_generation_module_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13d0ee715209614e"}},{"code_sha256_prefix":"c937fb5a4df322d0","entry":"get_chiron_simplification_format_prompt","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/chiron_simplification_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/chiron_simplification_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c937fb5a4df322d0"}},{"code_sha256_prefix":"a81c1a53e2d45ada","entry":"get_final_boolean_entailment_question","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/thought_chain_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/thought_chain_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a81c1a53e2d45ada"}},{"code_sha256_prefix":"0ec444a145f8dcc3","entry":"get_final_entailment_question","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/thought_chain_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/thought_chain_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ec444a145f8dcc3"}},{"code_sha256_prefix":"1811631193a7c1e1","entry":"get_instruction_preamble","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/prompt_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/prompt_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1811631193a7c1e1"}},{"code_sha256_prefix":"6575b299beb3916a","entry":"get_num_message_words","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/setup_rl_data.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/setup_rl_data.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6575b299beb3916a"}},{"code_sha256_prefix":"16a7e904fc8e6d4e","entry":"get_section_text","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/thought_chain_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/thought_chain_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16a7e904fc8e6d4e"}},{"code_sha256_prefix":"f599d52d4323a49c","entry":"get_story_section","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/prompt_utils.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/prompt_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f599d52d4323a49c"}},{"code_sha256_prefix":"18ed61dcfadbbdf0","entry":"reward_fn","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"rl_training/train_ncp.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/rl_training/train_ncp.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18ed61dcfadbbdf0"}},{"code_sha256_prefix":"2660750f248fdc8d","entry":"shorten_text_to_max_len","repo":"Alex-Gurung/ReasoningNCP","repo_kind":"official","path":"setup_data/compile_dataset.py","file_url":"https://github.com/Alex-Gurung/ReasoningNCP/blob/HEAD/setup_data/compile_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2660750f248fdc8d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}