{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/douzero-mastering-doudizhu-with-self-play","title":"DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning","arxiv_id":"2106.06135","date":"2021-06-11","proceeding":null,"authors":["Daochen Zha","Jingru Xie","Wenye Ma","Sheng Zhang","Xiangru Lian","Xia Hu","Ji Liu"],"abstract":"Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect- and imperfect-information games, DouDizhu (a.k.a. Fighting the Landlord), a three-player card game, is still unsolved. DouDizhu is a very challenging domain with competition, collaboration, imperfect information, large state space, and particularly a massive set of possible actions where the legal actions vary significantly from turn to turn. Unfortunately, modern reinforcement learning algorithms mainly focus on simple and small action spaces, and not surprisingly, are shown not to make satisfactory progress in DouDizhu. In this work, we propose a conceptually simple yet effective DouDizhu AI system, namely DouZero, which enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. Starting from scratch in a single server with four GPUs, DouZero outperformed all the existing DouDizhu AI programs in days of training and was ranked the first in the Botzone leaderboard among 344 AI agents. Through building DouZero, we show that classic Monte-Carlo methods can be made to deliver strong results in a hard domain with a complex action space. The code and an online demo are released at https://github.com/kwai/DouZero with the hope that this insight could motivate future work.","url_abs":"https://arxiv.org/abs/2106.06135v1","url_pdf":"https://arxiv.org/pdf/2106.06135v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"douzero-mastering-doudizhu-with-self-play","repo_url":"https://github.com/kwai/DouZero","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"card-games","task_name":"Card Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"game-of-poker","task_name":"Game of Poker"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"douzero","method_name":"DouZero"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[{"slug":"douzero","name":"DouZero","full_name":"DouZero"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.06135","atlas_url":"https://app.syntology.ai/?focus=2106.06135","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.06135"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kwai/DouZero","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1},"community":{"samples":3,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3998e0632bc7b7de","entry":"FarmerLstmModel","repo":"kwai/DouZero","repo_kind":"official","path":"douzero/dmc/models.py","file_url":"https://github.com/kwai/DouZero/blob/HEAD/douzero/dmc/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3998e0632bc7b7de"}},{"code_sha256_prefix":"dfccfb88738de17d","entry":"Model","repo":"kwai/DouZero","repo_kind":"official","path":"douzero/dmc/models.py","file_url":"https://github.com/kwai/DouZero/blob/HEAD/douzero/dmc/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfccfb88738de17d"}},{"code_sha256_prefix":"aefba852d67c0aec","entry":"get_latest_model","repo":"EdwardPooh/douzero-resnet-2.0","repo_kind":"community","path":"Douzero_Resnet/auto_test.py","file_url":"https://github.com/EdwardPooh/douzero-resnet-2.0/blob/HEAD/Douzero_Resnet/auto_test.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"aefba852d67c0aec"}},{"code_sha256_prefix":"d2b55c75f6fdae82","entry":"LandlordLstmModel","repo":"kwai/DouZero","repo_kind":"official","path":"douzero/dmc/models.py","file_url":"https://github.com/kwai/DouZero/blob/HEAD/douzero/dmc/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d2b55c75f6fdae82"}},{"code_sha256_prefix":"7a21191505af1ea4","entry":"compute_loss","repo":"Liberations/DouZero_For_HLDDZ_FullAuto","repo_kind":"community","path":"douzero/dmc/dmc.py","file_url":"https://github.com/Liberations/DouZero_For_HLDDZ_FullAuto/blob/HEAD/douzero/dmc/dmc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7a21191505af1ea4"}},{"code_sha256_prefix":"ee16d8c3495cc1e3","entry":"learn","repo":"Liberations/DouZero_For_HLDDZ_FullAuto","repo_kind":"community","path":"douzero/dmc/dmc.py","file_url":"https://github.com/Liberations/DouZero_For_HLDDZ_FullAuto/blob/HEAD/douzero/dmc/dmc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee16d8c3495cc1e3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}