Papers › Data Distillation: Towards Omni-Supervised Learning

Data Distillation: Towards Omni-Supervised Learning

12 Dec 2017CVPR 2018 6arXiv:1712.04440archive 2025-07-28

Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, Kaiming He

We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

EliPassov/classification-ensembles mentioned on GitHubpytorchSyntology: not harvested report
facebookresearch/detectron mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (arXiv:1506.01497; that paper's own run record: 79 ran (of which 9 constructed an object rather than computing a result; 47 with no instrument failure: 18 honoured, 5 violated, 24 with no contract checked; 32 where Syntology's instrument failed) · 45 unverified), Feature Pyramid Networks for Object Detection (arXiv:1612.03144; that paper's own run record: 33 ran (of which 9 constructed an object rather than computing a result; 28 with no instrument failure: 4 honoured, 0 violated, 24 with no contract checked; 5 where Syntology's instrument failed) · 18 unverified), Mask R-CNN (arXiv:1703.06870; that paper's own run record: 101 ran (of which 3 constructed an object rather than computing a result; 77 with no instrument failure: 2 honoured, 1 violated, 74 with no contract checked; 24 where Syntology's instrument failed) · 39 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
jiajunhua/facebookresearch-Detectron mentioned on GitHubcaffe2Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (arXiv:1506.01497; that paper's own run record: 79 ran (of which 9 constructed an object rather than computing a result; 47 with no instrument failure: 18 honoured, 5 violated, 24 with no contract checked; 32 where Syntology's instrument failed) · 45 unverified), Deep Residual Learning for Image Recognition (arXiv:1512.03385; that paper's own run record: 254 ran (of which 108 constructed an object rather than computing a result; 163 with no instrument failure: 3 honoured, 1 violated, 159 with no contract checked; 91 where Syntology's instrument failed) · 123 unverified), Aggregated Residual Transformations for Deep Neural Networks (arXiv:1611.05431; that paper's own run record: 51 ran (of which 7 constructed an object rather than computing a result; 31 with no instrument failure: 0 honoured, 0 violated, 31 with no contract checked; 20 where Syntology's instrument failed) · 29 unverified) and 2 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
notha99y/mean_teacher_domain_adapt mentioned on GitHubSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Temporal Ensembling for Semi-Supervised Learning (arXiv:1610.02242; that paper's own run record: 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Keypoint DetectionObject Detectionobject-detection

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 ConvolutionConvolutionFPNFaster R-CNNMask R-CNNRPNRoIAlignRoIPoolSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections