{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalizing-across-multi-objective-reward","title":"Generalizing Across Multi-Objective Reward Functions in Deep Reinforcement Learning","arxiv_id":"1809.06364","date":"2018-09-17","proceeding":null,"authors":["Eli Friedman","Fred Fontaine"],"abstract":"Many reinforcement-learning researchers treat the reward function as a part\nof the environment, meaning that the agent can only know the reward of a state\nif it encounters that state in a trial run. However, we argue that this is an\nunnecessary limitation and instead, the reward function should be provided to\nthe learning algorithm. The advantage is that the algorithm can then use the\nreward function to check the reward for states that the agent hasn't even\nencountered yet. In addition, the algorithm can simultaneously learn policies\nfor multiple reward functions. For each state, the algorithm would calculate\nthe reward using each of the reward functions and add the rewards to its\nexperience replay dataset. The Hindsight Experience Replay algorithm developed\nby Andrychowicz et al. (2017) does just this, and learns to generalize across a\ndistribution of sparse, goal-based rewards. We extend this algorithm to\nlinearly-weighted, multi-objective rewards and learn a single policy that can\ngeneralize across all linear combinations of the multi-objective reward.\nWhereas other multi-objective algorithms teach the Q-function to generalize\nacross the reward weights, our algorithm enables the policy to generalize, and\ncan thus be used with continuous actions.","url_abs":"http://arxiv.org/abs/1809.06364v1","url_pdf":"http://arxiv.org/pdf/1809.06364v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalizing-across-multi-objective-reward","repo_url":"https://github.com/trality/fire","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"experience-replay","method_name":"Experience Replay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.06364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1809.06364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/trality/fire","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"listed":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d13de08311608710","entry":"alr_output","repo":"trality/fire","repo_kind":"listed","path":"src/environment/environmentRL.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/environment/environmentRL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d13de08311608710"}},{"code_sha256_prefix":"9a0282092a5e7066","entry":"load_json","repo":"trality/fire","repo_kind":"listed","path":"src/util.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a0282092a5e7066"}},{"code_sha256_prefix":"7e8069e20ecde2e7","entry":"load_pickle","repo":"trality/fire","repo_kind":"listed","path":"src/util.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e8069e20ecde2e7"}},{"code_sha256_prefix":"6f9e6b00c5f3da97","entry":"lookback_array","repo":"trality/fire","repo_kind":"listed","path":"src/q_learning_agent.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/q_learning_agent.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f9e6b00c5f3da97"}},{"code_sha256_prefix":"af36d26d754c1dd2","entry":"mean_over_std","repo":"trality/fire","repo_kind":"listed","path":"src/environment/environment.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/environment/environment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af36d26d754c1dd2"}},{"code_sha256_prefix":"ce9d12e3161ca135","entry":"mean_over_std","repo":"trality/fire","repo_kind":"listed","path":"src/environment/environmentRL.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/environment/environmentRL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce9d12e3161ca135"}},{"code_sha256_prefix":"fc686fc120e24d48","entry":"over_t_ratio","repo":"trality/fire","repo_kind":"listed","path":"src/environment/environmentRL.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/environment/environmentRL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc686fc120e24d48"}},{"code_sha256_prefix":"414e19a6cdcf0706","entry":"sample_binomial","repo":"trality/fire","repo_kind":"listed","path":"src/q_learning_agent.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/q_learning_agent.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"414e19a6cdcf0706"}},{"code_sha256_prefix":"40fbaae9c4eb67a7","entry":"v2d","repo":"trality/fire","repo_kind":"listed","path":"src/util.py","file_url":"https://github.com/trality/fire/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"40fbaae9c4eb67a7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}