{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/leti-learning-to-generate-from-textual","title":"LeTI: Learning to Generate from Textual Interactions","arxiv_id":"2305.10314","date":"2023-05-17","proceeding":null,"authors":["Xingyao Wang","Hao Peng","Reyhaneh Jabbarvand","Heng Ji"],"abstract":"Fine-tuning pre-trained language models (LMs) is essential for enhancing their capabilities. Existing techniques commonly fine-tune on input-output pairs (e.g., instruction tuning) or with numerical rewards that gauge the output quality (e.g., RLHF). We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. This setting invites a natural and scalable way to acquire textual feedback: the error messages and stack traces from code execution using a Python interpreter. LETI iteratively fine-tunes the model, using the LM objective, on a concatenation of natural language instructions, LM-generated programs, and textual feedback. Prepended to this fine-tuning text, a binary reward token is used to differentiate correct and buggy solutions. LETI requires no ground-truth outputs for training and even outperforms a fine-tuned baseline that does. LETI not only improves the performance of LMs on a code generation dataset MBPP, but also generalizes to other datasets. Trained on MBPP, it achieves comparable or better performance than the base LMs on unseen problems in HumanEval. Furthermore, compared to binary feedback, we observe that textual feedback leads to improved generation quality and sample efficiency, achieving the same performance with fewer than half of the gradient steps. LETI is equally applicable in natural language tasks when they can be formulated as code generation, which we empirically verified on event argument extraction.","url_abs":"https://arxiv.org/abs/2305.10314v2","url_pdf":"https://arxiv.org/pdf/2305.10314v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"leti-learning-to-generate-from-textual","repo_url":"https://github.com/xingyaoww/leti","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"event-argument-extraction","task_name":"Event Argument Extraction"},{"task_slug":"humaneval","task_name":"HumanEval"},{"task_slug":null,"task_name":"mbpp"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.10314","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.10314"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xingyaoww/leti","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":11,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0f14840c40641281","entry":"apply_rotary_pos_emb","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/models/codegen/modeling_flax_codegen.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/models/codegen/modeling_flax_codegen.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f14840c40641281"}},{"code_sha256_prefix":"ff574bc6676f34f1","entry":"create_sinusoidal_positions","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/models/codegen/modeling_flax_codegen.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/models/codegen/modeling_flax_codegen.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff574bc6676f34f1"}},{"code_sha256_prefix":"c083e60f5001bd38","entry":"get_pairs","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/models/codegen/tokenization_codegen.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/models/codegen/tokenization_codegen.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c083e60f5001bd38"}},{"code_sha256_prefix":"768565e8f56c5fb0","entry":"rotate_every_two","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/models/codegen/modeling_flax_codegen.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/models/codegen/modeling_flax_codegen.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"768565e8f56c5fb0"}},{"code_sha256_prefix":"61055d376b415759","entry":"bce_loss_fn","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/common.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61055d376b415759"}},{"code_sha256_prefix":"aefddbe85377ea68","entry":"bce_loss_weighted_fn","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/common.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aefddbe85377ea68"}},{"code_sha256_prefix":"8203f70a8d1a3a21","entry":"gae_advantages","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/ppo.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8203f70a8d1a3a21"}},{"code_sha256_prefix":"b035bf1f5d63df47","entry":"gae_advantages_jax","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/ppo.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b035bf1f5d63df47"}},{"code_sha256_prefix":"34d602c3086874a6","entry":"mse_loss_fn","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/common.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34d602c3086874a6"}},{"code_sha256_prefix":"217714c5bbe2b144","entry":"ppo_objective","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/ppo.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"217714c5bbe2b144"}},{"code_sha256_prefix":"2c596ae865c6c476","entry":"reward_weighted_finetuning","repo":"xingyaoww/leti","repo_kind":"official","path":"leti/utils/algo/rw_finetune.py","file_url":"https://github.com/xingyaoww/leti/blob/HEAD/leti/utils/algo/rw_finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c596ae865c6c476"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}