{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/discrete-and-continuous-action-representation","title":"Discrete and Continuous Action Representation for Practical RL in Video Games","arxiv_id":"1912.11077","date":"2019-12-23","proceeding":null,"authors":["Olivier Delalleau","Maxim Peter","Eloi Alonso","Adrien Logut"],"abstract":"While most current research in Reinforcement Learning (RL) focuses on improving the performance of the algorithms in controlled environments, the use of RL under constraints like those met in the video game industry is rarely studied. Operating under such constraints, we propose Hybrid SAC, an extension of the Soft Actor-Critic algorithm able to handle discrete, continuous and parameterized actions in a principled way. We show that Hybrid SAC can successfully solve a highspeed driving task in one of our games, and is competitive with the state-of-the-art on parameterized actions benchmark tasks. We also explore the impact of using normalizing flows to enrich the expressiveness of the policy at minimal computational cost, and identify a potential undesired effect of SAC when used with normalizing flows, that may be addressed by optimizing a different objective.","url_abs":"https://arxiv.org/abs/1912.11077v1","url_pdf":"https://arxiv.org/pdf/1912.11077v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"discrete-and-continuous-action-representation","repo_url":"https://github.com/nisheeth-golakiya/hybrid-sac","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"control-with-prametrised-actions","task_name":"Control with Prametrised Actions"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"soft-actor-critic-autotuned-temperature","method_name":"Soft Actor-Critic (Autotuned Temperature)"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/control-with-prametrised-actions-on-half","task":"Control with Prametrised Actions","dataset":"Half Field Offence","model":"Hybrid SAC","rank_in_archive_order":2,"of":2,"metrics":{"Goal Probability":"0.639"},"uses_additional_data":false},{"leaderboard":"/sota/control-with-prametrised-actions-on-platform","task":"Control with Prametrised Actions","dataset":"Platform","model":"Hybrid SAC","rank_in_archive_order":2,"of":2,"metrics":{"Return":"0.981"},"uses_additional_data":false},{"leaderboard":"/sota/control-with-prametrised-actions-on-robot","task":"Control with Prametrised Actions","dataset":"Robot Soccer Goal","model":"Hybrid SAC","rank_in_archive_order":2,"of":2,"metrics":{"Goal Probability":"0.728"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1912.11077","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.11077"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nisheeth-golakiya/hybrid-sac","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f71e026482abff0b","entry":"gym_to_buffer","repo":"nisheeth-golakiya/hybrid-sac","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/nisheeth-golakiya/hybrid-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f71e026482abff0b"}},{"code_sha256_prefix":"cd6a98cbc5c7c67e","entry":"to_gym_action","repo":"nisheeth-golakiya/hybrid-sac","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/nisheeth-golakiya/hybrid-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd6a98cbc5c7c67e"}},{"code_sha256_prefix":"476ed7d3658ed66f","entry":"to_torch_action","repo":"nisheeth-golakiya/hybrid-sac","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/nisheeth-golakiya/hybrid-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"476ed7d3658ed66f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}