{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simba-simplicity-bias-for-scaling-up","title":"SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning","arxiv_id":"2410.09754","date":"2024-10-13","proceeding":null,"authors":["Hojoon Lee","Dongyoon Hwang","Donghu Kim","Hyunseung Kim","Jun Jet Tai","Kaushik Subramanian","Peter R. Wurman","Jaegul Choo","Peter Stone","Takuma Seno"],"abstract":"Recent advances in CV and NLP have been largely driven by scaling up the number of network parameters, despite traditional theories suggesting that larger networks are prone to overfitting. These large networks avoid overfitting by integrating components that induce a simplicity bias, guiding models toward simple and generalizable solutions. However, in deep RL, designing and scaling up networks have been less explored. Motivated by this opportunity, we present SimBa, an architecture designed to scale up parameters in deep RL by injecting a simplicity bias. SimBa consists of three components: (i) an observation normalization layer that standardizes inputs with running statistics, (ii) a residual feedforward block to provide a linear pathway from the input to output, and (iii) a layer normalization to control feature magnitudes. By scaling up parameters with SimBa, the sample efficiency of various deep RL algorithms-including off-policy, on-policy, and unsupervised methods-is consistently improved. Moreover, solely by integrating SimBa architecture into SAC, it matches or surpasses state-of-the-art deep RL methods with high computational efficiency across DMC, MyoSuite, and HumanoidBench. These results demonstrate SimBa's broad applicability and effectiveness across diverse RL algorithms and environments.","url_abs":"https://arxiv.org/abs/2410.09754v1","url_pdf":"https://arxiv.org/pdf/2410.09754v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simba-simplicity-bias-for-scaling-up","repo_url":"https://github.com/sonyresearch/simba","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"simba-simplicity-bias-for-scaling-up","repo_url":"https://github.com/MindSpore-scientific/code-8/tree/main/Simba","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"simba-simplicity-bias-for-scaling-up","repo_url":"https://github.com/araffin/sbx","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null},{"paper_slug":"simba-simplicity-bias-for-scaling-up","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/7/Simba","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"sac","method_name":"SAC"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.09754","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.09754"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sonyresearch/simba","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/araffin/sbx","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/7/Simba","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific/code-8/tree/main/Simba","reach":null}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1},"listed":{"samples":4,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c92989c54b17674e","entry":"ResidualBlock","repo":"sonyresearch/simba","repo_kind":"official","path":"scale_rl/networks/layers.py","file_url":"https://github.com/sonyresearch/simba/blob/HEAD/scale_rl/networks/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c92989c54b17674e"}},{"code_sha256_prefix":"8921b382ef8cb19c","entry":"SimbaResidualBlock","repo":"araffin/sbx","repo_kind":"listed","path":"sbx/common/jax_layers.py","file_url":"https://github.com/araffin/sbx/blob/HEAD/sbx/common/jax_layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8921b382ef8cb19c"}},{"code_sha256_prefix":"67336ee21b053fe2","entry":"he_normal_init","repo":"sonyresearch/simba","repo_kind":"official","path":"scale_rl/networks/layers.py","file_url":"https://github.com/sonyresearch/simba/blob/HEAD/scale_rl/networks/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"67336ee21b053fe2"}},{"code_sha256_prefix":"16298c972234921e","entry":"generate_sample","repo":"MindSpore-scientific/code-8","repo_kind":"listed","path":"QuantExplainNLP_AlgoAnalysis/data_preprocessing.py","file_url":"https://github.com/MindSpore-scientific/code-8/blob/HEAD/QuantExplainNLP_AlgoAnalysis/data_preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16298c972234921e"}},{"code_sha256_prefix":"5919a74fc358ee01","entry":"get_feature_names","repo":"MindSpore-scientific/code-8","repo_kind":"listed","path":"QuantExplainNLP_AlgoAnalysis/explainability.py","file_url":"https://github.com/MindSpore-scientific/code-8/blob/HEAD/QuantExplainNLP_AlgoAnalysis/explainability.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5919a74fc358ee01"}},{"code_sha256_prefix":"ef93260b5137f73b","entry":"maybe_num_nodes","repo":"MindSpore-scientific/code-8","repo_kind":"listed","path":"RGNN/model_simple.py","file_url":"https://github.com/MindSpore-scientific/code-8/blob/HEAD/RGNN/model_simple.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef93260b5137f73b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}