Papers › Can We Generate Shellcodes via Natural Language? An Empirical Study
Can We Generate Shellcodes via Natural Language? An Empirical Study
Pietro Liguori, Erfan Al-Hossami, Domenico Cotroneo, Roberto Natella, Bojan Cukic, Samira Shaikh
Writing software exploits is an important practice for offensive security analysts to investigate and prevent attacks. In particular, shellcodes are especially time-consuming and a technical challenge, as they are written in assembly language. In this work, we address the task of automatically generating shellcodes, starting purely from descriptions in natural language, by proposing an approach based on Neural Machine Translation (NMT). We then present an empirical study using a novel dataset (Shellcode_IA32), which consists of 3,200 assembly code snippets of real Linux/x86 shellcodes from public databases, annotated using natural language. Moreover, we propose novel metrics to evaluate the accuracy of NMT at generating shellcodes. The empirical analysis shows that NMT can generate assembly code snippets from the natural language with high accuracy and that in many cases can generate entire shellcodes with no errors.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Code Generation | Shellcode_IA32 | CodeBERT | BLEU-4 | 91.70 | #1 of 3 | Archive leaderboard | report |
| Code Generation | Shellcode_IA32 | CodeBERT | Exact Match Accuracy | 89.75 | #1 of 3 | Archive leaderboard | report |
| Code Generation | Shellcode_IA32 | Seq2Seq with Attention | BLEU-4 | 90.03 | #2 of 3 | Archive leaderboard | report |
| Code Generation | Shellcode_IA32 | Seq2Seq with Attention | Exact Match Accuracy | 82.92 | #2 of 3 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections