can-dataset

The controller area network (CAN) bus has emerged as the de facto standard for in-vehicle networks (IVNs) around the globe. Safety-critical components (e.g., the brakes, the engine, the transmission) depend on the CAN bus for expedient, reliable communication. Unfortunately, while the CAN bus was designed to be resilient under harsh operating conditions, it was not designed to be resilient under adversarial conditions. Standard security practices such as authentication, authorization, and encryption are completely lacking when it comes to the CAN bus. Researchers have since developed authentication, authorization, and encryption specifications for the CAN bus, but retroactive implementation of said security controls would be exorbitantly expensive—in terms of hardware, labor, engineering effort, and monetary cost. Therefore, the automotive intrusion detection system (IDS) has emerged in the literature as a low-cost, low-effort solution to the automotive [in]security problem. However, developing and evaluating an automotive IDS can be quite challenging—especially if researchers lack access to a test vehicle. Without a test vehicle, researchers are limited to publicly available CAN data, and existing CAN intrusion detection datasets come with various limitations. This lack of CAN data has become a barrier to entry into automotive intrusion detection research—and even automotive security research in general.

We seek to lower this barrier to entry by introducing a new CAN intrusion detection dataset, which facilitates the development and evaluation of automotive IDSs. Our dataset, can-train-and-test, offers real-world CAN traffic data from four different vehicles—a sedan, a compact SUV, a full-size SUV, and a pickup truck—produced by two different manufacturers. For each vehicle, we provided comparable attack captures, which enable researchers to assess a given IDS's ability to generalize to different vehicle types and models. Our dataset contains .log files for playback as well as labeled and unlabeled .csv files for supervised and unsupervised machine learning. As such, our dataset is well suited to a variety of different automotive intrusion detection and automotive security enterprises. In addition, can-train-and-test supplies nine unique attacks, ranging from denial of service (DoS) fuzzing to triple spoofing attacks. As such, researchers can select from a wide variety of attacks when partitioning the data into training and testing datasets. Alternatively, researchers can leverage our curated can-train-and-test repository, which is subdivided into four train/test sub-datasets and four testing subsets. As a benchmark, we pitted 18 machine learning models against the can-train-and-test repository. During our evaluation and analysis, we found that the multi-layer perceptron, gradient boosting, isolation forest, BIRCH, and logistic regression models consistently scored above 0.95 when it came to accuracy, precision, recall, and F1-score—regardless of the sub-dataset and testing subset. Across all experiments on all sub-datasets, we saw an average F1-score of ≈0.5039, indicating that our can-train-and-test dataset is indeed capable of distinguishing capable, well-trained IDSs from their less-than-capable counterparts. We present can-train-and-test as a contribution to the existing collection of open-access CAN intrusion detection datasets in hopes of filling in the gaps left by the existing collection.