{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/xdo-a-double-oracle-algorithm-for-extensive","title":"XDO: A Double Oracle Algorithm for Extensive-Form Games","arxiv_id":"2103.06426","date":"2021-03-11","proceeding":"NeurIPS 2021 12","authors":["Stephen Mcaleer","John Lanier","Kevin Wang","Pierre Baldi","Roy Fox"],"abstract":"Policy Space Response Oracles (PSRO) is a reinforcement learning (RL) algorithm for two-player zero-sum games that has been empirically shown to find approximate Nash equilibria in large games. Although PSRO is guaranteed to converge to an approximate Nash equilibrium and can handle continuous actions, it may take an exponential number of iterations as the number of information states (infostates) grows. We propose Extensive-Form Double Oracle (XDO), an extensive-form double oracle algorithm for two-player zero-sum games that is guaranteed to converge to an approximate Nash equilibrium linearly in the number of infostates. Unlike PSRO, which mixes best responses at the root of the game, XDO mixes best responses at every infostate. We also introduce Neural XDO (NXDO), where the best response is learned through deep RL. In tabular experiments on Leduc poker, we find that XDO achieves an approximate Nash equilibrium in a number of iterations an order of magnitude smaller than PSRO. Experiments on a modified Leduc poker game and Oshi-Zumo show that tabular XDO achieves a lower exploitability than CFR with the same amount of computation. We also find that NXDO outperforms PSRO and NFSP on a sequential multidimensional continuous-action game. NXDO is the first deep RL method that can find an approximate Nash equilibrium in high-dimensional continuous-action sequential games. Experiment code is available at https://github.com/indylab/nxdo.","url_abs":"https://arxiv.org/abs/2103.06426v2","url_pdf":"https://arxiv.org/pdf/2103.06426v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"xdo-a-double-oracle-algorithm-for-extensive","repo_url":"https://github.com/indylab/nxdo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"form","task_name":"Form"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.06426","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.06426"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/indylab/nxdo","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1},"community":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ff24c0265ef0bddd","entry":"SafeFallbackJSONEncoder","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff24c0265ef0bddd"}},{"code_sha256_prefix":"8ee2503f180b5135","entry":"StrategySpec","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ee2503f180b5135"}},{"code_sha256_prefix":"c0b3a0e916709ae1","entry":"check_if_jsonable","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0b3a0e916709ae1"}},{"code_sha256_prefix":"ffff3a51188e0637","entry":"datetime_str","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffff3a51188e0637"}},{"code_sha256_prefix":"d29981ec0da3193d","entry":"policy_class_instance_to_full_policy","repo":"indylab/tabular_xdo","repo_kind":"community","path":"xdo_psro_fixed_population_comparison.py","file_url":"https://github.com/indylab/tabular_xdo/blob/HEAD/xdo_psro_fixed_population_comparison.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d29981ec0da3193d"}},{"code_sha256_prefix":"83aab9bc5f690c88","entry":"NXDOManager","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83aab9bc5f690c88"}},{"code_sha256_prefix":"40e9050b70dc2e70","entry":"RestrictedGameSolveResult","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"40e9050b70dc2e70"}},{"code_sha256_prefix":"e82865b6d2eed8be","entry":"SolveRestrictedGame","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e82865b6d2eed8be"}},{"code_sha256_prefix":"c670869610afca4c","entry":"ensure_dir","repo":"indylab/nxdo","repo_kind":"official","path":"grl/algos/nxdo/nxdo_manager/manager.py","file_url":"https://github.com/indylab/nxdo/blob/HEAD/grl/algos/nxdo/nxdo_manager/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c670869610afca4c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}