Papers › Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag...
Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag Challenges: Trade-offs between Model-free Learning and A Priori Knowledge
Fabio Massimo Zennaro, Laszlo Erdodi
Penetration testing is a security exercise aimed at assessing the security of a system by simulating attacks against it. So far, penetration testing has been carried out mainly by trained human attackers and its success critically depended on the available expertise. Automating this practice constitutes a non-trivial problem, as the range of actions that a human expert may attempts against a system and the range of knowledge she relies on to take her decisions are hard to capture. In this paper, we focus our attention on simplified penetration testing problems expressed in the form of capture the flag hacking challenges, and we analyze how model-free reinforcement learning algorithms may help to solve them. In modeling these capture the flag competitions as reinforcement learning problems we highlight that a specific challenge that characterize penetration testing is the problem of discovering the structure of the problem at hand. We then show how this challenge may be eased by relying on different forms of prior knowledge that may be provided to the agent. In this way we demonstrate how the feasibility of tackling penetration testing using reinforcement learning may rest on a careful trade-off between model-free and model-based algorithms. By using techniques to inject a priori knowledge, we show it is possible to better direct the agent and restrict the space of its exploration problem, thus achieving solutions more efficiently.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections