{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/de-rrd-a-knowledge-distillation-framework-for","title":"DE-RRD: A Knowledge Distillation Framework for Recommender System","arxiv_id":"2012.04357","date":"2020-12-08","proceeding":null,"authors":["SeongKu Kang","Junyoung Hwang","Wonbin Kweon","Hwanjo Yu"],"abstract":"Recent recommender systems have started to employ knowledge distillation, which is a model compression technique distilling knowledge from a cumbersome model (teacher) to a compact model (student), to reduce inference latency while maintaining performance. The state-of-the-art methods have only focused on making the student model accurately imitate the predictions of the teacher model. They have a limitation in that the prediction results incompletely reveal the teacher's knowledge. In this paper, we propose a novel knowledge distillation framework for recommender system, called DE-RRD, which enables the student model to learn from the latent knowledge encoded in the teacher model as well as from the teacher's predictions. Concretely, DE-RRD consists of two methods: 1) Distillation Experts (DE) that directly transfers the latent knowledge from the teacher model. DE exploits \"experts\" and a novel expert selection strategy for effectively distilling the vast teacher's knowledge to the student with limited capacity. 2) Relaxed Ranking Distillation (RRD) that transfers the knowledge revealed from the teacher's prediction with consideration of the relaxed ranking orders among items. Our extensive experiments show that DE-RRD outperforms the state-of-the-art competitors and achieves comparable or even better performance to that of the teacher model with faster inference time.","url_abs":"https://arxiv.org/abs/2012.04357v1","url_pdf":"https://arxiv.org/pdf/2012.04357v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"de-rrd-a-knowledge-distillation-framework-for","repo_url":"https://github.com/SeongKu-Kang/DE-RRD_CIKM20","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"de-rrd-a-knowledge-distillation-framework-for","repo_url":"https://github.com/dudwns511/dcd_cikm21","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.04357","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.04357"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dudwns511/dcd_cikm21","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SeongKu-Kang/DE-RRD_CIKM20","reach":{"status":"unanswered"}}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"9872645d835762b4","entry":"relaxed_ranking_loss","repo":"dudwns511/dcd_cikm21","repo_kind":"listed","path":"Utils/dcd_loss.py","file_url":"https://github.com/dudwns511/dcd_cikm21/blob/HEAD/Utils/dcd_loss.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9872645d835762b4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}