{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-reward-adjustment-in-multi-reward","title":"Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation","arxiv_id":"2403.13578","date":"2024-03-20","proceeding":null,"authors":["Do June Min","Veronica Perez-Rosas","Kenneth Resnicow","Rada Mihalcea"],"abstract":"In this paper, we study the problem of multi-reward reinforcement learning to jointly optimize for multiple text qualities for natural language generation. We focus on the task of counselor reflection generation, where we optimize the generators to simultaneously improve the fluency, coherence, and reflection quality of generated counselor responses. We introduce two novel bandit methods, DynaOpt and C-DynaOpt, which rely on the broad strategy of combining rewards into a single value and optimizing them simultaneously. Specifically, we employ non-contextual and contextual multi-arm bandits to dynamically adjust multiple reward weights during training. Through automatic and manual evaluations, we show that our proposed techniques, DynaOpt and C-DynaOpt, outperform existing naive and bandit baselines, showcasing their potential for enhancing language models.","url_abs":"https://arxiv.org/abs/2403.13578v1","url_pdf":"https://arxiv.org/pdf/2403.13578v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-reward-adjustment-in-multi-reward","repo_url":"https://github.com/michigannlp/dynaopt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.13578","atlas_url":"https://app.syntology.ai/?focus=2403.13578","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.13578"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michigannlp/dynaopt","reach":null}],"summary":{"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d68fe00f92e5fbb7","entry":"to_vw_example_format","repo":"michigannlp/dynaopt","repo_kind":"official","path":"con_rl_train.py","file_url":"https://github.com/michigannlp/dynaopt/blob/HEAD/con_rl_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d68fe00f92e5fbb7"}},{"code_sha256_prefix":"7f9d7f4332331eda","entry":"sample_custom_pmf","repo":"michigannlp/dynaopt","repo_kind":"official","path":"con_rl_train.py","file_url":"https://github.com/michigannlp/dynaopt/blob/HEAD/con_rl_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f9d7f4332331eda"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}