{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/no-press-diplomacy-from-scratch","title":"No-Press Diplomacy from Scratch","arxiv_id":"2110.02924","date":"2021-10-06","proceeding":"NeurIPS 2021 12","authors":["Anton Bakhtin","David Wu","Adam Lerer","Noam Brown"],"abstract":"Prior AI successes in complex games have largely focused on settings with at most hundreds of actions at each decision point. In contrast, Diplomacy is a game with more than 10^20 possible actions per turn. Previous attempts to address games with large branching factors, such as Diplomacy, StarCraft, and Dota, used human data to bootstrap the policy or used handcrafted reward shaping. In this paper, we describe an algorithm for action exploration and equilibrium approximation in games with combinatorial action spaces. This algorithm simultaneously performs value iteration while learning a policy proposal network. A double oracle step is used to explore additional actions to add to the policy proposals. At each state, the target state value and policy for the model training are computed via an equilibrium search procedure. Using this algorithm, we train an agent, DORA, completely from scratch for a popular two-player variant of Diplomacy and show that it achieves superhuman performance. Additionally, we extend our methods to full-scale no-press Diplomacy and for the first time train an agent from scratch with no human data. We present evidence that this agent plays a strategy that is incompatible with human-data bootstrapped agents. This presents the first strong evidence of multiple equilibria in Diplomacy and suggests that self play alone may be insufficient for achieving superhuman performance in Diplomacy.","url_abs":"https://arxiv.org/abs/2110.02924v1","url_pdf":"https://arxiv.org/pdf/2110.02924v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"no-press-diplomacy-from-scratch","repo_url":"https://github.com/facebookresearch/diplomacy_searchbot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"starcraft","task_name":"Starcraft"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.02924","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.02924"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/diplomacy_searchbot","reach":null}],"summary":{"ran_draft_wrong":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"708b1574dcdcc04c","entry":"apply_batch_repeat_interleave","repo":"facebookresearch/diplomacy_searchbot","repo_kind":"official","path":"fairdiplomacy/models/diplomacy_model/diplomacy_model.py","file_url":"https://github.com/facebookresearch/diplomacy_searchbot/blob/HEAD/fairdiplomacy/models/diplomacy_model/diplomacy_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"708b1574dcdcc04c"}},{"code_sha256_prefix":"dafac81b1a5b0905","entry":"repeat_interleave_if_tensor","repo":"facebookresearch/diplomacy_searchbot","repo_kind":"official","path":"fairdiplomacy/models/diplomacy_model/diplomacy_model.py","file_url":"https://github.com/facebookresearch/diplomacy_searchbot/blob/HEAD/fairdiplomacy/models/diplomacy_model/diplomacy_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dafac81b1a5b0905"}},{"code_sha256_prefix":"7fbf1eb777211f41","entry":"compute_alignments","repo":"facebookresearch/diplomacy_searchbot","repo_kind":"official","path":"fairdiplomacy/models/diplomacy_model/diplomacy_model.py","file_url":"https://github.com/facebookresearch/diplomacy_searchbot/blob/HEAD/fairdiplomacy/models/diplomacy_model/diplomacy_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fbf1eb777211f41"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}