Im4Sketch

Im4Sketch is a large-scale dataset with shape-oriented set of classes for image-to-sketch generalization . It consists of a collection of natural images from 874 categories for training and validation, and sketches from 393 categories (a subset of natural image categories) for testing.

The images and sketches are collected from existing popular computer vision datasets. The categories are selected having shape similarity in mind, so that object with same shape belong to the same category.

The natural-image part of the dataset is based on the ILSVRC2012 version of ImageNet. The original ImageNet categories are first merged according to the shape criteria. Object categories for objects whose shape, e.g. how a human would draw the object, is the same are merged. For this step, semantic similarity of categories, obtained through WordNet and category names, is used to obtain candidate categories for merging. Based on visual inspection of these candidates, the decision to merge the original ImageNet classes is made by a human. For instance, ”Indian Elephant” and ”African Elephant”, or ”Laptop” and ”Notebook” are merged. An extreme case of merging is the new class “dog” that is a union of 121 original ImageNet classes of dog breeds.

In the second step, classes from datasets containing sketches are used. In particular, DomainNet, Sketchy, PACS, and TU-Berlin. Note that merging is not necessary for classes in these datasets, because the shape criteria are guaranteed since they are designed for sketches. In this step, a correspondence between the merged ImageNet categories and categories of the other datasets is found. As in the merging step, semantic similarity is used to guide the correspondence search. Sketch categories that are not present in the merged ImageNet are added to the overall category set, while training natural images of those categories are collected from either DomainNet or Sketchy. In the end, ImageNet is used for 690 classes, DomainNet for 183 classes, and Sketchy for 1 class, respectively.