{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/knowledge-adaptation-for-efficient-semantic","title":"Knowledge Adaptation for Efficient Semantic Segmentation","arxiv_id":"1903.04688","date":"2019-03-12","proceeding":"CVPR 2019 6","authors":["Tong He","Chunhua Shen","Zhi Tian","Dong Gong","Changming Sun","Youliang Yan"],"abstract":"Both accuracy and efficiency are of significant importance to the task of\nsemantic segmentation. Existing deep FCNs suffer from heavy computations due to\na series of high-resolution feature maps for preserving the detailed knowledge\nin dense estimation. Although reducing the feature map resolution (i.e.,\napplying a large overall stride) via subsampling operations (e.g., pooling and\nconvolution striding) can instantly increase the efficiency, it dramatically\ndecreases the estimation accuracy. To tackle this dilemma, we propose a\nknowledge distillation method tailored for semantic segmentation to improve the\nperformance of the compact FCNs with large overall stride. To handle the\ninconsistency between the features of the student and teacher network, we\noptimize the feature similarity in a transferred latent domain formulated by\nutilizing a pre-trained autoencoder. Moreover, an affinity distillation module\nis proposed to capture the long-range dependency by calculating the non-local\ninteractions across the whole image. To validate the effectiveness of our\nproposed method, extensive experiments have been conducted on three popular\nbenchmarks: Pascal VOC, Cityscapes and Pascal Context. Built upon a highly\ncompetitive baseline, our proposed method can improve the performance of a\nstudent network by 2.5\\% (mIOU boosts from 70.2 to 72.7 on the cityscapes test\nset) and can train a better compact model with only 8\\% float operations\n(FLOPS) of a model that achieves comparable performances.","url_abs":"http://arxiv.org/abs/1903.04688v1","url_pdf":"http://arxiv.org/pdf/1903.04688v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"knowledge-adaptation-for-efficient-semantic","repo_url":"https://github.com/yoshitomo-matsubara/torchdistill","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.04688","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}