Papers › Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

3 Mar 2016arXiv:1603.01121archive 2025-07-28

Johannes Heinrich, David Silver

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

EricSteinberger/DREAM mentioned on GitHubMITSyntology: not harvested report
IAARhub/TrucoAnalytics mentioned on GitHubSyntology: not harvested report
TinkeringCode/Neural-Fictitous-Self-Play mentioned on GitHubpytorchMITSyntology: not harvested report
heidekrueger/bnelearn mentioned on GitHubpytorchGPL-3.0Syntology: not harvested report
jsanderink/tue mentioned on GitHubpytorchMITSyntology: not harvested report
quantumiracle/mars mentioned on GitHubpytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in... (arXiv:2207.08894; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report
deepmind/open_spiel Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers (arXiv:2106.09435; that paper's own run record: 9 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 9 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Card GamesDeep Reinforcement LearningGame of PokerReinforcement LearningReinforcement Learning (RL)reinforcement-learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections