{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/harmony-world-models-boosting-sample","title":"HarmonyDream: Task Harmonization Inside World Models","arxiv_id":"2310.00344","date":"2023-09-30","proceeding":null,"authors":["Haoyu Ma","Jialong Wu","Ningya Feng","Chenjun Xiao","Dong Li","Jianye Hao","Jianmin Wang","Mingsheng Long"],"abstract":"Model-based reinforcement learning (MBRL) holds the promise of sample-efficient learning by utilizing a world model, which models how the environment works and typically encompasses components for two tasks: observation modeling and reward modeling. In this paper, through a dedicated empirical investigation, we gain a deeper understanding of the role each task plays in world models and uncover the overlooked potential of sample-efficient MBRL by mitigating the domination of either observation or reward modeling. Our key insight is that while prevalent approaches of explicit MBRL attempt to restore abundant details of the environment via observation models, it is difficult due to the environment's complexity and limited model capacity. On the other hand, reward models, while dominating implicit MBRL and adept at learning compact task-centric dynamics, are inadequate for sample-efficient learning without richer learning signals. Motivated by these insights and discoveries, we propose a simple yet effective approach, HarmonyDream, which automatically adjusts loss coefficients to maintain task harmonization, i.e. a dynamic equilibrium between the two tasks in world model learning. Our experiments show that the base MBRL method equipped with HarmonyDream gains 10%-69% absolute performance boosts on visual robotic tasks and sets a new state-of-the-art result on the Atari 100K benchmark. Code is available at https://github.com/thuml/HarmonyDream.","url_abs":"https://arxiv.org/abs/2310.00344v3","url_pdf":"https://arxiv.org/pdf/2310.00344v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"harmony-world-models-boosting-sample","repo_url":"https://github.com/thuml/harmonydream","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"atari-games-100k","task_name":"Atari Games 100k"},{"task_slug":"model-based-reinforcement-learning","task_name":"Model-based Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.00344","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.00344"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/thuml/HarmonyDream","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thuml/harmonydream","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"24fe13b4f847da6b","entry":"cast_to_compute","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/jaxutils.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/jaxutils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24fe13b4f847da6b"}},{"code_sha256_prefix":"bbd25fa32893729e","entry":"pure","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/ninjax.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/ninjax.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bbd25fa32893729e"}},{"code_sha256_prefix":"e5b1910ea05891b1","entry":"subsample","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/jaxutils.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/jaxutils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5b1910ea05891b1"}},{"code_sha256_prefix":"67c6d2355562342f","entry":"tensorstats","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/jaxutils.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/jaxutils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67c6d2355562342f"}},{"code_sha256_prefix":"03dd934385b4d240","entry":"get_act","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/nets.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/nets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03dd934385b4d240"}},{"code_sha256_prefix":"1ad869363216c281","entry":"grad","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/ninjax.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/ninjax.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ad869363216c281"}},{"code_sha256_prefix":"45b97b51433849a7","entry":"rng","repo":"thuml/HarmonyDream","repo_kind":"official","path":"dreamerv3-jax/dreamerv3/ninjax.py","file_url":"https://github.com/thuml/HarmonyDream/blob/HEAD/dreamerv3-jax/dreamerv3/ninjax.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45b97b51433849a7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}