{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/storm-efficient-stochastic-transformer-based-1","title":"STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning","arxiv_id":"2310.09615","date":"2023-10-14","proceeding":"NeurIPS 2023 11","authors":["Weipu Zhang","Gang Wang","Jian Sun","Yetian Yuan","Gao Huang"],"abstract":"Recently, model-based reinforcement learning algorithms have demonstrated remarkable efficacy in visual input environments. These approaches begin by constructing a parameterized simulation world model of the real environment through self-supervised learning. By leveraging the imagination of the world model, the agent's policy is enhanced without the constraints of sampling from the real environment. The performance of these algorithms heavily relies on the sequence modeling and generation capabilities of the world model. However, constructing a perfectly accurate model of a complex unknown environment is nearly impossible. Discrepancies between the model and reality may cause the agent to pursue virtual goals, resulting in subpar performance in the real environment. Introducing random noise into model-based reinforcement learning has been proven beneficial. In this work, we introduce Stochastic Transformer-based wORld Model (STORM), an efficient world model architecture that combines the strong sequence modeling and generation capabilities of Transformers with the stochastic nature of variational autoencoders. STORM achieves a mean human performance of $126.7\\%$ on the Atari $100$k benchmark, setting a new record among state-of-the-art methods that do not employ lookahead search techniques. Moreover, training an agent with $1.85$ hours of real-time interaction experience on a single NVIDIA GeForce RTX 3090 graphics card requires only $4.3$ hours, showcasing improved efficiency compared to previous methodologies.","url_abs":"https://arxiv.org/abs/2310.09615v1","url_pdf":"https://arxiv.org/pdf/2310.09615v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"storm-efficient-stochastic-transformer-based-1","repo_url":"https://github.com/weipu-zhang/storm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"atari-games-100k","task_name":"Atari Games 100k"},{"task_slug":"model-based-reinforcement-learning","task_name":"Model-based Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.09615","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.09615"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weipu-zhang/storm","reach":{"status":"ok"}}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"ffb0647ebfd42a95","entry":"calc_lambda_return","repo":"weipu-zhang/storm","repo_kind":"official","path":"agents.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/agents.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffb0647ebfd42a95"}},{"code_sha256_prefix":"6ae6a81b313635f6","entry":"get_subsequent_mask","repo":"weipu-zhang/storm","repo_kind":"official","path":"sub_models/attention_blocks.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/sub_models/attention_blocks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ae6a81b313635f6"}},{"code_sha256_prefix":"e38bff42d47263cd","entry":"get_subsequent_mask_with_batch_length","repo":"weipu-zhang/storm","repo_kind":"official","path":"sub_models/attention_blocks.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/sub_models/attention_blocks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e38bff42d47263cd"}},{"code_sha256_prefix":"3c26bddfecd514dd","entry":"get_vector_mask","repo":"weipu-zhang/storm","repo_kind":"official","path":"sub_models/attention_blocks.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/sub_models/attention_blocks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c26bddfecd514dd"}},{"code_sha256_prefix":"b6471f7466453c3c","entry":"load_config","repo":"weipu-zhang/storm","repo_kind":"official","path":"utils.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b6471f7466453c3c"}},{"code_sha256_prefix":"adce9ddcc08da8dc","entry":"percentile","repo":"weipu-zhang/storm","repo_kind":"official","path":"agents.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/agents.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"adce9ddcc08da8dc"}},{"code_sha256_prefix":"67b3a332ec394684","entry":"symexp","repo":"weipu-zhang/storm","repo_kind":"official","path":"sub_models/functions_losses.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/sub_models/functions_losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67b3a332ec394684"}},{"code_sha256_prefix":"c5f6f10f82106e10","entry":"symlog","repo":"weipu-zhang/storm","repo_kind":"official","path":"sub_models/functions_losses.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/sub_models/functions_losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5f6f10f82106e10"}},{"code_sha256_prefix":"b81345145f0ea196","entry":"build_single_env","repo":"weipu-zhang/storm","repo_kind":"official","path":"env_wrapper.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/env_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b81345145f0ea196"}},{"code_sha256_prefix":"dbdef7d52f9792a4","entry":"build_vec_env","repo":"weipu-zhang/storm","repo_kind":"official","path":"env_wrapper.py","file_url":"https://github.com/weipu-zhang/storm/blob/HEAD/env_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbdef7d52f9792a4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}