{"url":"/dataset/edge-iiotset","name":"EDGE-IIOTSET","full_name":"A NEW COMPREHENSIVE REALISTIC CYBER SECURITY DATASET OF IOT AND IIOT APPLICATIONS: CENTRALIZED AND FEDERATED LEARNING","description_markdown":"ABSTRACT \r\nIn this project, we propose a new comprehensive realistic cyber security dataset of IoT and IIoT applications, called Edge-IIoTset, which can be used by machine learning-based intrusion detection systems in two different modes, namely, centralized and federated learning. Specifically, the proposed testbed is organized into seven layers, including, Cloud Computing Layer, Network Functions Virtualization Layer, Blockchain Network Layer, Fog Computing Layer, Software-Defined Networking Layer, Edge Computing Layer, and IoT and IIoT Perception Layer. In each layer, we propose new emerging technologies that satisfy the key requirements of IoT and IIoT applications, such as, ThingsBoard IoT platform, OPNFV platform, Hyperledger Sawtooth, Digital twin, ONOS SDN controller, Mosquitto MQTT brokers, Modbus TCP/IP, ...etc. The IoT data are generated from various IoT devices (more than 10 types) such as Low-cost digital sensors for sensing temperature and humidity, Ultrasonic sensor, Water level detection sensor, pH Sensor Meter, Soil Moisture sensor, Heart Rate Sensor, Flame Sensor, ...etc.). However, we identify and analyze fourteen attacks related to IoT and IIoT connectivity protocols, which are categorized into five threats, including, DoS/DDoS attacks, Information gathering, Man in the middle attacks, Injection attacks, and Malware attacks. In addition, we extract features obtained from different sources, including alerts, system resources, logs, network traffic, and propose new 61 features with high correlations from 1176 found features. After processing and analyzing the proposed realistic cyber security dataset, we provide a primary exploratory data analysis and evaluate the performance of machine learning approaches (i.e., traditional machine learning as well as deep learning) in both centralized and federated learning modes.\r\n\r\nInstructions: \r\n \r\n\r\nGreat news! The Edge-IIoT dataset has been featured as a \"Document in the top 1% of Web of Science.\" This indicates that it is ranked within the top 1% of all publications indexed by the Web of Science (WoS) in terms of citations and impact.\r\n\r\nPlease kindly visit kaggle link for the updates: https://www.kaggle.com/datasets/mohamedamineferrag/edgeiiotset-cyber-sec...\r\n\r\n \r\n\r\nFree use of the Edge-IIoTset dataset for academic research purposes is hereby granted in perpetuity. Use for commercial purposes is allowable after asking the leader author, Dr Mohamed Amine Ferrag, who has asserted his right under the Copyright.\r\n\r\nThe details of the Edge-IIoT dataset were published in following the paper. For the academic/public use of these datasets, the authors have to cities the following paper:\r\n\r\nMohamed Amine Ferrag, Othmane Friha, Djallel Hamouda, Leandros Maglaras, Helge Janicke, \"Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning\", IEEE Access, April 2022 (IF: 3.37), DOI: 10.1109/ACCESS.2022.3165809\r\n\r\nLink to paper : https://ieeexplore.ieee.org/document/9751703\r\n\r\n********************************************\r\n\r\nThe directories of the Edge-IIoTset dataset include the following:\r\n\r\n•File 1 (Normal traffic)\r\n\r\n-File 1.1 (Distance): This file includes two documents, namely, Distance.csv and Distance.pcap. The IoT sensor (Ultrasonic sensor) is used to capture the IoT data.\r\n\r\n-File 1.2 (Flame_Sensor): This file includes two documents, namely, Flame_Sensor.csv and Flame_Sensor.pcap. The IoT sensor (Flame Sensor) is used to capture the IoT data.\r\n\r\n-File 1.3 (Heart_Rate): This file includes two documents, namely, Flame_Sensor.csv and Flame_Sensor.pcap. The IoT sensor (Flame Sensor) is used to capture the IoT data.\r\n\r\n-File 1.4 (IR_Receiver): This file includes two documents, namely, IR_Receiver.csv and IR_Receiver.pcap. The IoT sensor (IR (Infrared) Receiver Sensor) is used to capture the IoT data.\r\n\r\n-File 1.5 (Modbus): This file includes two documents, namely, Modbus.csv and Modbus.pcap. The IoT sensor (Modbus Sensor) is used to capture the IoT data.\r\n\r\n-File 1.6 (phValue): This file includes two documents, namely, phValue.csv and phValue.pcap. The IoT sensor (pH-sensor PH-4502C) is used to capture the IoT data.\r\n\r\n-File 1.7 (Soil_Moisture): This file includes two documents, namely, Soil_Moisture.csv and Soil_Moisture.pcap. The IoT sensor (Soil Moisture Sensor v1.2) is used to capture the IoT data.\r\n\r\n-File 1.8 (Sound_Sensor): This file includes two documents, namely, Sound_Sensor.csv and Sound_Sensor.pcap. The IoT sensor (LM393 Sound Detection Sensor) is used to capture the IoT data.\r\n\r\n-File 1.9 (Temperature_and_Humidity): This file includes two documents, namely, Temperature_and_Humidity.csv and Temperature_and_Humidity.pcap. The IoT sensor (DHT11 Sensor) is used to capture the IoT data.\r\n\r\n-File 1.10 (Water_Level): This file includes two documents, namely, Water_Level.csv and Water_Level.pcap. The IoT sensor (Water sensor) is used to capture the IoT data.\r\n\r\n•File 2 (Attack traffic): \r\n\r\n-File 2.1 (Attack traffic (CSV files)): This file includes 13 documents, namely, Backdoor_attack.csv, DDoS_HTTP_Flood_attack.csv, DDoS_ICMP_Flood_attack.csv, DDoS_TCP_SYN_Flood_attack.csv, DDoS_UDP_Flood_attack.csv, MITM_attack.csv, OS_Fingerprinting_attack.csv, Password_attack.csv, Port_Scanning_attack.csv, Ransomware_attack.csv, SQL_injection_attack.csv, Uploading_attack.csv, Vulnerability_scanner_attack.csv, XSS_attack.csv. Each document is specific for each attack.\r\n\r\n-File 2.2 (Attack traffic (PCAP files)): This file includes 13 documents, namely, Backdoor_attack.pcap, DDoS_HTTP_Flood_attack.pcap, DDoS_ICMP_Flood_attack.pcap, DDoS_TCP_SYN_Flood_attack.pcap, DDoS_UDP_Flood_attack.pcap, MITM_attack.pcap, OS_Fingerprinting_attack.pcap, Password_attack.pcap, Port_Scanning_attack.pcap, Ransomware_attack.pcap, SQL_injection_attack.pcap, Uploading_attack.pcap, Vulnerability_scanner_attack.pcap, XSS_attack.pcap. Each document is specific for each attack.\r\n\r\n•File 3 (Selected dataset for ML and DL): \r\n\r\n-File 3.1 (DNN-EdgeIIoT-dataset): This file contains a selected dataset for the use of evaluating deep learning-based intrusion detection systems. \r\n\r\n-File 3.2 (ML-EdgeIIoT-dataset): This file contains a selected dataset for the use of evaluating traditional machine learning-based intrusion detection systems. \r\n\r\n********************************************\r\n\r\nStep 1: Downloading The Edge-IIoTset dataset From the Kaggle platform\r\nfrom google.colab import files\r\n\r\n!pip install -q kaggle\r\n\r\nfiles.upload()\r\n\r\n!mkdir ~/.kaggle\r\n\r\n!cp kaggle.json ~/.kaggle/\r\n\r\n!chmod 600 ~/.kaggle/kaggle.json\r\n\r\n!kaggle datasets download -d mohamedamineferrag/edgeiiotset-cyber-security-dataset-of-iot-iiot -f \"Edge-IIoTset dataset/Selected dataset for ML and DL/DNN-EdgeIIoT-dataset.csv\"\r\n\r\n!unzip DNN-EdgeIIoT-dataset.csv.zip\r\n\r\n!rm DNN-EdgeIIoT-dataset.csv.zip\r\n\r\nStep 2: Reading the Datasets' CSV file to a Pandas DataFrame:\r\nimport pandas as pd\r\n\r\nimport numpy as np\r\n\r\ndf = pd.read_csv('DNN-EdgeIIoT-dataset.csv', low_memory=False) \r\n\r\n Step 3 : Exploring some of the DataFrame's contents:\r\ndf.head(5)\r\n\r\nprint(df['Attack_type'].value_counts())\r\n\r\nStep 4: Dropping data (Columns, duplicated rows, NAN, Null..):\r\nfrom sklearn.utils import shuffle\r\n\r\ndrop_columns = [\"frame.time\", \"ip.src_host\", \"ip.dst_host\", \"arp.src.proto_ipv4\",\"arp.dst.proto_ipv4\", \r\n\r\n         \"http.file_data\",\"http.request.full_uri\",\"icmp.transmit_timestamp\",\r\n\r\n         \"http.request.uri.query\", \"tcp.options\",\"tcp.payload\",\"tcp.srcport\",\r\n\r\n         \"tcp.dstport\", \"udp.port\", \"mqtt.msg\"]\r\n\r\ndf.drop(drop_columns, axis=1, inplace=True)\r\n\r\ndf.dropna(axis=0, how='any', inplace=True)\r\n\r\ndf.drop_duplicates(subset=None, keep=\"first\", inplace=True)\r\n\r\ndf = shuffle(df)\r\n\r\ndf.isna().sum()\r\n\r\nprint(df['Attack_type'].value_counts())\r\n\r\nStep 5: Categorical data encoding (Dummy Encoding):\r\nimport numpy as np\r\n\r\nfrom sklearn.model_selection import train_test_split\r\n\r\nfrom sklearn.preprocessing import StandardScaler\r\n\r\nfrom sklearn import preprocessing\r\n\r\ndef encode_text_dummy(df, name):\r\n\r\n    dummies = pd.get_dummies(df[name])\r\n\r\n    for x in dummies.columns:\r\n\r\n        dummy_name = f\"{name}-{x}\"\r\n\r\n        df[dummy_name] = dummies[x]\r\n\r\n    df.drop(name, axis=1, inplace=True)\r\n\r\nencode_text_dummy(df,'http.request.method')\r\n\r\nencode_text_dummy(df,'http.referer')\r\n\r\nencode_text_dummy(df,\"http.request.version\")\r\n\r\nencode_text_dummy(df,\"dns.qry.name.len\")\r\n\r\nencode_text_dummy(df,\"mqtt.conack.flags\")\r\n\r\nencode_text_dummy(df,\"mqtt.protoname\")\r\n\r\nencode_text_dummy(df,\"mqtt.topic\")\r\n\r\nStep 6: Creation of the preprocessed dataset\r\ndf.to_csv('preprocessed_DNN.csv', encoding='utf-8')\r\n\r\n********************************************\r\n\r\nFor more information about the dataset, please contact the lead author of this project, Dr Mohamed Amine Ferrag, on his email: mohamed.amine.ferrag@gmail.com \r\n\r\nMore information about Dr. Mohamed Amine Ferrag is available at:\r\n\r\nhttps://www.linkedin.com/in/Mohamed-Amine-Ferrag \r\n\r\nhttps://dblp.uni-trier.de/pid/142/9937.html \r\n\r\nhttps://www.researchgate.net/profile/Mohamed_Amine_Ferrag \r\n\r\nhttps://scholar.google.fr/citations?user=IkPeqxMAAAAJ&hl=fr&oi=ao \r\n\r\nhttps://www.scopus.com/authid/detail.uri?authorId=56115001200 \r\n\r\nhttps://publons.com/researcher/1322865/mohamed-amine-ferrag/ \r\n\r\nhttps://orcid.org/0000-0002-0632-3172 \r\n\r\n \r\n\r\nLast Updated: 27 Mar. 2023","description_withheld":null,"homepage":"https://ieee-dataport.org/documents/edge-iiotset-new-comprehensive-realistic-cyber-security-dataset-iot-and-iiot-applications","introduced_date":null,"introduced_date_note":null,"introduced_by":null,"license":null,"modalities":[],"tasks":[{"name":"Intrusion Detection","url":"/task/intrusion-detection","datasets_with_task":"/datasets/task/intrusion-detection"},{"name":"Network Intrusion Detection","url":"/task/network-intrusion-detection","datasets_with_task":"/datasets/task/network-intrusion-detection"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["EDGE-IIOTSET"],"data_loaders":[],"num_papers_in_archive":4,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}