NuiSI Dataset (Nuitrack Skeleton Interaction Dataset)

The NuiSI dataset contains skeleton tracking trajectories of Human Interaction Partners performing a variety of physically interactive behaviors (waving, handshaking, rocket fistbump, parachute fistbump) with each other. This is inspired by the dataset in Bütepage et al. "Imitating by generating: Deep generative models for imitation of interactive tasks." Frontiers in Robotics and AI (2020) wherein they capture a dataset with rokoko motion capture suits. Instead we track the skeletons of the interaction partner with Intel Realsense cameras using Nuitrack, for a more realistic scenario, with noise coming from the depth sensor, the skeleton tracking and some partial occlusions. This makes it more representative of real world interactions with a Robot equipped with an RGBD camera. T This dataset is used in our papers for training Interaction models for Human-Robot Interaction with a humanoid social robot. If you find the dataset useful in your work, please cite our paper:

Prasad, V., Heitlinger, L., Koert, D., Stock-Homburg, R., Peters, J., & Chalvatzaki, G. (2023). Learning multimodal latent dynamics for human-robot interaction. arXiv preprint arXiv:2311.16380.

@article{prasad2023learning, title={Learning multimodal latent dynamics for human-robot interaction}, author={Prasad, Vignesh and Heitlinger, Lea and Koert, Dorothea and Stock-Homburg, Ruth and Peters, Jan and Chalvatzaki, Georgia}, journal={arXiv preprint arXiv:2311.16380}, year={2023} }