{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-efficient-detector-with-semi","title":"Learning Efficient Detector with Semi-supervised Adaptive Distillation","arxiv_id":"1901.00366","date":"2019-01-02","proceeding":null,"authors":["Shitao Tang","Litong Feng","Wenqi Shao","Zhanghui Kuang","Wei zhang","Yimin Chen"],"abstract":"Knowledge Distillation (KD) has been used in image classification for model\ncompression. However, rare studies apply this technology on single-stage object\ndetectors. Focal loss shows that the accumulated errors of easily-classified\nsamples dominate the overall loss in the training process. This problem is also\nencountered when applying KD in the detection task. For KD, the teacher-defined\nhard samples are far more important than any others. We propose ADL to address\nthis issue by adaptively mimicking the teacher's logits, with more attention\npaid on two types of hard samples: hard-to-learn samples predicted by teacher\nwith low certainty and hard-to-mimic samples with a large gap between the\nteacher's and the student's prediction. ADL enlarges the distillation loss for\nhard-to-learn and hard-to-mimic samples and reduces distillation loss for the\ndominant easy samples, enabling distillation to work on the single-stage\ndetector first time, even if the student and the teacher are identical.\nBesides, ADL is effective in both the supervised setting and the\nsemi-supervised setting, even when the labeled data and unlabeled data are from\ndifferent distributions. For distillation on unlabeled data, ADL achieves\nbetter performance than existing data distillation which simply utilizes hard\ntargets, making the student detector surpass its teacher. On the COCO database,\nsemi-supervised adaptive distillation (SAD) makes a student detector with a\nbackbone of ResNet-50 surpasses its teacher with a backbone of ResNet-101,\nwhile the student has half of the teacher's computation complexity. The code is\navaiable at https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation","url_abs":"http://arxiv.org/abs/1901.00366v2","url_pdf":"http://arxiv.org/pdf/1901.00366v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-efficient-detector-with-semi","repo_url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"caffe2","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"focal-loss","method_name":"Focal Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.00366","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.00366"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"54e95d897362fc40","entry":"GetOpNodeProducer","repo":"Tangshitao/Semi-supervised-Adaptive-Distillation","repo_kind":"official","path":"caffe2/caffe2/python/net_drawer.py","file_url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation/blob/HEAD/caffe2/caffe2/python/net_drawer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54e95d897362fc40"}},{"code_sha256_prefix":"ed2e1bbef8cdf436","entry":"commonprefix","repo":"Tangshitao/Semi-supervised-Adaptive-Distillation","repo_kind":"official","path":"caffe2/caffe2/python/net_printer.py","file_url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation/blob/HEAD/caffe2/caffe2/python/net_printer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed2e1bbef8cdf436"}},{"code_sha256_prefix":"3707b7a05c1a29c7","entry":"factor_prefix","repo":"Tangshitao/Semi-supervised-Adaptive-Distillation","repo_kind":"official","path":"caffe2/caffe2/python/net_printer.py","file_url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation/blob/HEAD/caffe2/caffe2/python/net_printer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3707b7a05c1a29c7"}},{"code_sha256_prefix":"f1f9a4ac54736551","entry":"format_value","repo":"Tangshitao/Semi-supervised-Adaptive-Distillation","repo_kind":"official","path":"caffe2/caffe2/python/net_printer.py","file_url":"https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation/blob/HEAD/caffe2/caffe2/python/net_printer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1f9a4ac54736551"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}