{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/collaborative-distillation-for-ultra","title":"Collaborative Distillation for Ultra-Resolution Universal Style Transfer","arxiv_id":"2003.08436","date":"2020-03-18","proceeding":"CVPR 2020 6","authors":["Huan Wang","Yijun Li","Yuehai Wang","Haoji Hu","Ming-Hsuan Yang"],"abstract":"Universal style transfer methods typically leverage rich representations from deep Convolutional Neural Network (CNN) models (e.g., VGG-19) pre-trained on large collections of images. Despite the effectiveness, its application is heavily constrained by the large model size to handle ultra-resolution images given limited memory. In this work, we present a new knowledge distillation method (named Collaborative Distillation) for encoder-decoder based neural style transfer to reduce the convolutional filters. The main idea is underpinned by a finding that the encoder-decoder pairs construct an exclusive collaborative relationship, which is regarded as a new kind of knowledge for style transfer models. Moreover, to overcome the feature size mismatch when applying collaborative distillation, a linear embedding loss is introduced to drive the student network to learn a linear embedding of the teacher's features. Extensive experiments show the effectiveness of our method when applied to different universal style transfer approaches (WCT and AdaIN), even if the model size is reduced by 15.5 times. Especially, on WCT with the compressed models, we achieve ultra-resolution (over 40 megapixels) universal style transfer on a 12GB GPU for the first time. Further experiments on optimization-based stylization scheme show the generality of our algorithm on different stylization paradigms. Our code and trained models are available at https://github.com/mingsun-tse/collaborative-distillation.","url_abs":"https://arxiv.org/abs/2003.08436v2","url_pdf":"https://arxiv.org/pdf/2003.08436v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"collaborative-distillation-for-ultra","repo_url":"https://github.com/mingsun-tse/collaborative-distillation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"style-transfer","task_name":"Style Transfer"}],"methods":[{"method_slug":"collaborative-distillation","method_name":"Collaborative Distillation"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[{"slug":"collaborative-distillation","name":"Collaborative Distillation","full_name":"Collaborative Distillation"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.08436","atlas_url":"https://app.syntology.ai/?focus=2003.08436","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.08436"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mingsun-tse/collaborative-distillation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"16679a24b9881031","entry":"adaptive_instance_normalization","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"model/model_cd.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/model/model_cd.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16679a24b9881031"}},{"code_sha256_prefix":"38eaf8511dc7bc9c","entry":"calc_mean_std","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"model/model_cd.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/model/model_cd.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38eaf8511dc7bc9c"}},{"code_sha256_prefix":"c6160e81afad8c48","entry":"check_path","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"utils.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6160e81afad8c48"}},{"code_sha256_prefix":"ac269a0e4b8d946e","entry":"default_loader","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"PytorchWCT/data_loader.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/PytorchWCT/data_loader.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac269a0e4b8d946e"}},{"code_sha256_prefix":"b8ed90af599ee615","entry":"gram_matrix","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"model/model_cd.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/model/model_cd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b8ed90af599ee615"}},{"code_sha256_prefix":"77fc78d38d0f565f","entry":"is_img","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"utils.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77fc78d38d0f565f"}},{"code_sha256_prefix":"0516d5020cce56da","entry":"is_image_file","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"PytorchWCT/data_loader.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/PytorchWCT/data_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0516d5020cce56da"}},{"code_sha256_prefix":"b82dd11ae570eba4","entry":"set_up_dir","repo":"mingsun-tse/collaborative-distillation","repo_kind":"official","path":"utils.py","file_url":"https://github.com/mingsun-tse/collaborative-distillation/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b82dd11ae570eba4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}