3D-POP

The dataset is designed specifically to solve a range of computer vision problems (2D-3D tracking, posture) faced by biologists while designing behavior studies with animals.

Typically, datasets for animal-specific vision tasks are created using open-source video material. This might be effective for an initial start, but these methods are not deployment ready for the behavior community. Therefore, we designed a semi-automated method for biologists to create well-curated datasets at a large scale for the ML and Vision community.

3D-POP is the first dataset with 3D ground truth for multi-animal, multi-view tracking problems.

Highlight: The dataset is captured with the intention of using it for various vision problems and with different levels of complexity (no of cameras, no of individuals)

Video explanation: Link to YouTube video

Video teaser: Link to YouTube video

Dataset Features:

Marker-based videos:

• 6 hours+ of annotations of 18 individuals (groups of 1, 2, 5, 10).
• Bounding box
• Trajectories (2D and 3D)
• Posture (2D and 3D) with 9 key points
• Identities
• Total of 57 sequences (4K) with 4 views.
• Dataset customization* (Users can modify the dataset and add key points to the dataset)

Markerless:

• 1Hr+ videos of 18 individuals in groups of 1, 2, 5, 11. The birds have no markers. This data is provided as test cases and unsupervised approaches.

Problems:

2D domain:

• Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview.
• Tracking with single - multiview

3D domain:

• Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview.
• Tracking with single - multiview

Fine-grained recognition:

• Identity tracking with ground truth.

Unsupervised learning:

• 2D or 3D posture problems

Idea:

The dataset is created with a motion capture system, using the 6-DOF tracking ability. Assumptions are that head and body act as rigid bodies when birds walk and forage (proved with experiment). Therefore, we get the 3D position of key points by tracking head/body orientation.