{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ember-an-open-dataset-for-training-static-pe","title":"EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models","arxiv_id":"1804.04637","date":"2018-04-12","proceeding":null,"authors":["Hyrum S. Anderson","Phil Roth"],"abstract":"This paper describes EMBER: a labeled benchmark dataset for training machine learning models to statically detect malicious Windows portable executable files. The dataset includes features extracted from 1.1M binary files: 900K training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test samples (100K malicious, 100K benign). To accompany the dataset, we also release open source code for extracting features from additional binaries so that additional sample features can be appended to the dataset. This dataset fills a void in the information security machine learning community: a benign/malicious dataset that is large, open and general enough to cover several interesting use cases. We enumerate several use cases that we considered when structuring the dataset. Additionally, we demonstrate one use case wherein we compare a baseline gradient boosted decision tree model trained using LightGBM with default settings to MalConv, a recently published end-to-end (featureless) deep learning model for malware detection. Results show that even without hyper-parameter optimization, the baseline EMBER model outperforms MalConv. The authors hope that the dataset, code and baseline model provided by EMBER will help invigorate machine learning research for malware detection, in much the same way that benchmark datasets have advanced computer vision research.","url_abs":"http://arxiv.org/abs/1804.04637v2","url_pdf":"http://arxiv.org/pdf/1804.04637v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/endgameinc/ember","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/Abdullah-B/Best-Effort-Adversarial-Approximation-of-Black-Box-Malware-Classifiers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/BhaveshRaj27/Malware-detection-using-Deep-learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/dtrizna/nebula","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/dtrizna/quo.vadis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/elastic/ember","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/gxenos/EmberML","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/himanshujoshijiit/malware-detection-of-pefiles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/powerli2002/Transformer-on-ember","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/pralab/toucanstrike","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/sala-group/autows-bench-101","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"ember-an-open-dataset-for-training-static-pe","repo_url":"https://github.com/chaiyanyu17/malware-dataset","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[{"slug":"ember","name":"EMBER","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.04637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1804.04637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BhaveshRaj27/Malware-detection-using-Deep-learning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chaiyanyu17/malware-dataset","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dtrizna/nebula","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/powerli2002/Transformer-on-ember","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/endgameinc/ember","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Abdullah-B/Best-Effort-Adversarial-Approximation-of-Black-Box-Malware-Classifiers","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sala-group/autows-bench-101","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/himanshujoshijiit/malware-detection-of-pefiles","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dtrizna/quo.vadis","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elastic/ember","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gxenos/EmberML","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pralab/toucanstrike","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":2,"ran_violates":1,"unverified":2},"by_repo_kind":{"listed":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8429e6520e353d4d","entry":"batched_index_select","repo":"dtrizna/nebula","repo_kind":"listed","path":"nebula/models/reformer.py","file_url":"https://github.com/dtrizna/nebula/blob/HEAD/nebula/models/reformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8429e6520e353d4d"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"dtrizna/nebula","repo_kind":"listed","path":"nebula/models/reformer.py","file_url":"https://github.com/dtrizna/nebula/blob/HEAD/nebula/models/reformer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"8b42f84b228597d9","entry":"sort_key_val","repo":"dtrizna/nebula","repo_kind":"listed","path":"nebula/models/reformer.py","file_url":"https://github.com/dtrizna/nebula/blob/HEAD/nebula/models/reformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b42f84b228597d9"}},{"code_sha256_prefix":"0da2f30f9daf8d28","entry":"create_dataloader","repo":"dtrizna/nebula","repo_kind":"listed","path":"nebula/data_utils.py","file_url":"https://github.com/dtrizna/nebula/blob/HEAD/nebula/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0da2f30f9daf8d28"}},{"code_sha256_prefix":"8138cb55581c1b66","entry":"generate_square_subsequent_mask","repo":"dtrizna/nebula","repo_kind":"listed","path":"nebula/models/attention.py","file_url":"https://github.com/dtrizna/nebula/blob/HEAD/nebula/models/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8138cb55581c1b66"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}