{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distilling-object-detectors-with-fine-grained-1","title":"Distilling Object Detectors with Fine-grained Feature Imitation","arxiv_id":"1906.03609","date":"2019-06-09","proceeding":"CVPR 2019 6","authors":["Tao Wang","Li Yuan","Xiaopeng Zhang","Jiashi Feng"],"abstract":"State-of-the-art CNN based recognition models are often computationally prohibitive to deploy on low-end devices. A promising high level approach tackling this limitation is knowledge distillation, which let small student model mimic cumbersome teacher model's output to get improved generalization. However, related methods mainly focus on simple task of classification while do not consider complex tasks like object detection. We show applying the vanilla knowledge distillation to detection model gets minor gain. To address the challenge of distilling knowledge in detection model, we propose a fine-grained feature imitation method exploiting the cross-location discrepancy of feature response. Our intuition is that detectors care more about local near object regions. Thus the discrepancy of feature response on the near object anchor locations reveals important information of how teacher model tends to generalize. We design a novel mechanism to estimate those locations and let student model imitate the teacher on them to get enhanced performance. We first validate the idea on a developed lightweight toy detector which carries simplest notion of current state-of-the-art anchor based detection models on challenging KITTI dataset, our method generates up to 15% boost of mAP for the student model compared to the non-imitated counterpart. We then extensively evaluate the method with Faster R-CNN model under various scenarios with common object detection benchmark of Pascal VOC and COCO, imitation alleviates up to 74% performance drop of student model compared to teacher. Codes released at https://github.com/twangnh/Distilling-Object-Detectors","url_abs":"https://arxiv.org/abs/1906.03609v1","url_pdf":"https://arxiv.org/pdf/1906.03609v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distilling-object-detectors-with-fine-grained-1","repo_url":"https://github.com/twangnh/Distilling-Object-Detectors","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"distilling-object-detectors-with-fine-grained-1","repo_url":"https://github.com/p517332051/GAN-Knowledge-Distillation-SSD","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"distilling-object-detectors-with-fine-grained-1","repo_url":"https://github.com/wonbeomjang/yolov5-knowledge-distillation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"AGPL-3.0"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"faster-r-cnn","method_name":"Faster R-CNN"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"roipool","method_name":"RoIPool"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.03609","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.03609"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wonbeomjang/yolov5-knowledge-distillation","reach":{"status":"ok","spdx":"AGPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/p517332051/GAN-Knowledge-Distillation-SSD","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/twangnh/Distilling-Object-Detectors","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3be9228820b46ef3","entry":"get_imdb","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/factory.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3be9228820b46ef3"}},{"code_sha256_prefix":"0eafc51289a49033","entry":"parse_rec","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/voc_eval.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/voc_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0eafc51289a49033"}},{"code_sha256_prefix":"8a015c012f507631","entry":"unique_boxes","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/ds_utils.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/ds_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a015c012f507631"}},{"code_sha256_prefix":"c485a145dcf5de8e","entry":"vg_eval","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/vg_eval.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/vg_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c485a145dcf5de8e"}},{"code_sha256_prefix":"ec3e5aa793d6ce69","entry":"voc_ap","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/voc_eval.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/voc_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec3e5aa793d6ce69"}},{"code_sha256_prefix":"2f16ecec36118ba9","entry":"voc_eval","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/voc_eval.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/voc_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f16ecec36118ba9"}},{"code_sha256_prefix":"d3e019b11b709edb","entry":"xywh_to_xyxy","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/ds_utils.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/ds_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3e019b11b709edb"}},{"code_sha256_prefix":"a832e5017f504280","entry":"xyxy_to_xywh","repo":"twangnh/Distilling-Object-Detectors","repo_kind":"official","path":"lib/datasets/ds_utils.py","file_url":"https://github.com/twangnh/Distilling-Object-Detectors/blob/HEAD/lib/datasets/ds_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a832e5017f504280"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}