{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-scale-aligned-distillation-for-low-1","title":"Multi-Scale Aligned Distillation for Low-Resolution Detection","arxiv_id":"2109.06875","date":"2021-09-14","proceeding":"CVPR 2021 1","authors":["Lu Qi","Jason Kuen","Jiuxiang Gu","Zhe Lin","Yi Wang","Yukang Chen","Yanwei Li","Jiaya Jia"],"abstract":"In instance-level detection tasks (e.g., object detection), reducing input resolution is an easy option to improve runtime efficiency. However, this option traditionally hurts the detection performance much. This paper focuses on boosting the performance of low-resolution models by distilling knowledge from a high- or multi-resolution model. We first identify the challenge of applying knowledge distillation (KD) to teacher and student networks that act on different input resolutions. To tackle it, we explore the idea of spatially aligning feature maps between models of varying input resolutions by shifting feature pyramid positions and introduce aligned multi-scale training to train a multi-scale teacher that can distill its knowledge to a low-resolution student. Further, we propose crossing feature-level fusion to dynamically fuse teacher's multi-resolution features to guide the student better. On several instance-level detection tasks and datasets, the low-resolution models trained via our approach perform competitively with high-resolution models trained via conventional multi-scale training, while outperforming the latter's low-resolution models by 2.1% to 3.6% in terms of mAP. Our code is made publicly available at https://github.com/dvlab-research/MSAD.","url_abs":"https://arxiv.org/abs/2109.06875v1","url_pdf":"https://arxiv.org/pdf/2109.06875v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-scale-aligned-distillation-for-low-1","repo_url":"https://github.com/Jia-Research-Lab/MSAD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"multi-scale-aligned-distillation-for-low-1","repo_url":"https://github.com/dvlab-research/msad","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.06875","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.06875"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dvlab-research/msad","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Jia-Research-Lab/MSAD","reach":null}],"summary":{"ran_honours":1,"ran":3},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"6baff9bff3755d04","entry":"EPE","repo":"Jia-Research-Lab/MSAD","repo_kind":"official","path":"msad/layers/loss.py","file_url":"https://github.com/Jia-Research-Lab/MSAD/blob/HEAD/msad/layers/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6baff9bff3755d04"}},{"code_sha256_prefix":"5f22bd913308e42f","entry":"L1","repo":"Jia-Research-Lab/MSAD","repo_kind":"official","path":"msad/layers/loss.py","file_url":"https://github.com/Jia-Research-Lab/MSAD/blob/HEAD/msad/layers/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f22bd913308e42f"}},{"code_sha256_prefix":"b0e79cd4dedcd4e5","entry":"L2","repo":"Jia-Research-Lab/MSAD","repo_kind":"official","path":"msad/layers/loss.py","file_url":"https://github.com/Jia-Research-Lab/MSAD/blob/HEAD/msad/layers/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0e79cd4dedcd4e5"}},{"code_sha256_prefix":"0d75b5053b3b0641","entry":"MultiScale","repo":"Jia-Research-Lab/MSAD","repo_kind":"official","path":"msad/layers/loss.py","file_url":"https://github.com/Jia-Research-Lab/MSAD/blob/HEAD/msad/layers/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d75b5053b3b0641"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}