{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trust-the-model-where-it-trusts-itself-model","title":"Trust the Model Where It Trusts Itself -- Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaption","arxiv_id":"2405.19014","date":"2024-05-29","proceeding":null,"authors":["Bernd Frauenknecht","Artur Eisele","Devdutt Subhasish","Friedrich Solowjow","Sebastian Trimpe"],"abstract":"Dyna-style model-based reinforcement learning (MBRL) combines model-free agents with predictive transition models through model-based rollouts. This combination raises a critical question: 'When to trust your model?'; i.e., which rollout length results in the model providing useful data? Janner et al. (2019) address this question by gradually increasing rollout lengths throughout the training. While theoretically tempting, uniform model accuracy is a fallacy that collapses at the latest when extrapolating. Instead, we propose asking the question 'Where to trust your model?'. Using inherent model uncertainty to consider local accuracy, we obtain the Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaption (MACURA) algorithm. We propose an easy-to-tune rollout mechanism and demonstrate substantial improvements in data efficiency and performance compared to state-of-the-art deep MBRL methods on the MuJoCo benchmark.","url_abs":"https://arxiv.org/abs/2405.19014v3","url_pdf":"https://arxiv.org/pdf/2405.19014v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trust-the-model-where-it-trusts-itself-model","repo_url":"https://github.com/Data-Science-in-Mechanical-Engineering/macura","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"model-based-reinforcement-learning","task_name":"Model-based Reinforcement Learning"},{"task_slug":"mujoco","task_name":"MuJoCo"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.19014","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.19014"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Data-Science-in-Mechanical-Engineering/macura","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bc872cf944838841","entry":"get_cnn_output_size","repo":"Data-Science-in-Mechanical-Engineering/macura","repo_kind":"official","path":"mbrl/models/util.py","file_url":"https://github.com/Data-Science-in-Mechanical-Engineering/macura/blob/HEAD/mbrl/models/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc872cf944838841"}},{"code_sha256_prefix":"74092aa5844e5a7b","entry":"get_masking_rate","repo":"Data-Science-in-Mechanical-Engineering/macura","repo_kind":"official","path":"mbrl/algorithms/m2ac.py","file_url":"https://github.com/Data-Science-in-Mechanical-Engineering/macura/blob/HEAD/mbrl/algorithms/m2ac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74092aa5844e5a7b"}},{"code_sha256_prefix":"2870c393c5ce3bdf","entry":"calc_uncertainty_score","repo":"Data-Science-in-Mechanical-Engineering/macura","repo_kind":"official","path":"mbrl/algorithms/m2ac.py","file_url":"https://github.com/Data-Science-in-Mechanical-Engineering/macura/blob/HEAD/mbrl/algorithms/m2ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2870c393c5ce3bdf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}