EEG Speech-Robot Interaction Dataset (EEG data recorded during spoken and imagined speech interaction with a simulated robot)

Dataset Description

This dataset consists of Electroencephalography (EEG) data recorded from 15 healthy subjects using a 64-channel EEG headset during spoken and imagined speech interaction with a simulated robot.

Citation

The dataset recording and study setup are described in detail in the following publications:

1. Rekrut, M., Selim, A. M., & Krüger, A. (2022, October). Improving Silent Speech BCI Training Procedures Through Transfer from Overt to Silent Speech. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2650–2656. IEEE. https://doi.org/10.1109/SMC53654.2022.9945447
2. Selim, A. M., Rekrut, M., Barz, M., & Sonntag, D. (2024, June). Speech Imagery BCI Training Using Game with a Purpose. In Proceedings of the 2024 International Conference on Advanced Visual Interfaces (AVI ’24). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3656650.3656654

If you use this dataset, please consider citing these works.

Study Design

Participants controlled a simulated robot on a screen in a game-like setup through a maze while EEG data was recorded. Below are the key details:

Experimental Setup

• Interaction Modes:
• Overt Speech: Speaking command words aloud.

• Imagined Speech: Silently imagining the command words without any articulatory muscle movement.

• Command Words:

• Movement: “Left”, “Right”, “Up”

• Actions: “Pick”, “Push”

• Interaction Process:

• Participants pressed the spacebar to indicate readiness.
• The screen turned black for 2 seconds to prepare input.
• A fixation cross appeared, signalling participants to speak or imagine the desired command.
• The fixation cross disappeared, and the robot’s position updated accordingly.

• The robot always performed the correct action, ensuring participants were stress-free.

• Session Organisation:

• Four parts: Overt Speech, Imagined Speech, Overt Speech, Imagined Speech.
• 80 repetitions per word and paradigm (400 imagined, 400 spoken per participant).
• Task divided into levels of 25 interactions to reduce cognitive load.
• Breaks between sessions allowed for rest and EEG headset impedance checks.

Participants

• Subjects: 15 healthy individuals (11 male, 4 female), average age 26.8 years.
• Attributes: Right-handed, fluent in English (non-native).
• Ethics: Informed consent obtained; approved by the ethical review board of the Faculty of Mathematics and Computer Science, Saarland University.

Recording Conditions

• Environment: Dimly lit room with minimal distractions.
• EEG Equipment: Wireless 64-channel Brain Products Live Amp 64 system.
• Sampling rate: 500 Hz.
• Electrode placement: 10-20 International System.
• Synchronisation: The robot game and EEG recording software were executed on the same computer.

Data Format

• File Format: .fif format, containing a continuous raw data stream with event markers.
• Event Dictionary:
  ```json { "Event_Dictionary": { "Empty": 1, "EndOfEvent": 2, "EndOfLevel": 3, "EndOfParadigm": 4, "space_bar": 5, "Overt_Up": 11, "Overt_Left": 12, "Overt_Right": 13, "Overt_Pick": 14, "Overt_Push": 15, "Silent_Up": 21, "Silent_Left": 22, "Silent_Right": 23, "Silent_Pick": 24, "Silent_Push": 25 } }