{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dual-student-breaking-the-limits-of-the","title":"Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning","arxiv_id":"1909.01804","date":"2019-09-03","proceeding":"ICCV 2019 10","authors":["Zhanghan Ke","Daoye Wang","Qiong Yan","Jimmy Ren","Rynson W. H. Lau"],"abstract":"Recently, consistency-based methods have achieved state-of-the-art results in semi-supervised learning (SSL). These methods always involve two roles, an explicit or implicit teacher model and a student model, and penalize predictions under different perturbations by a consistency constraint. However, the weights of these two roles are tightly coupled since the teacher is essentially an exponential moving average (EMA) of the student. In this work, we show that the coupled EMA teacher causes a performance bottleneck. To address this problem, we introduce Dual Student, which replaces the teacher with another student. We also define a novel concept, stable sample, following which a stabilization constraint is designed for our structure to be trainable. Further, we discuss two variants of our method, which produce even higher performance. Extensive experiments show that our method improves the classification performance significantly on several main SSL benchmarks. Specifically, it reduces the error rate of the 13-layer CNN from 16.84% to 12.39% on CIFAR-10 with 1k labels and from 34.10% to 31.56% on CIFAR-100 with 10k labels. In addition, our method also achieves a clear improvement in domain adaptation.","url_abs":"https://arxiv.org/abs/1909.01804v1","url_pdf":"https://arxiv.org/pdf/1909.01804v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dual-student-breaking-the-limits-of-the","repo_url":"https://github.com/ZHKKKe/DualStudent","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"dual-student-breaking-the-limits-of-the","repo_url":"https://github.com/60972823l/SSL-DNLL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"semi-supervised-image-classification","task_name":"Semi-Supervised Image Classification"},{"task_slug":"unsupervised-domain-adaptation","task_name":"Unsupervised Domain Adaptation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semi-supervised-image-classification-on-cifar-11","task":"Semi-Supervised Image Classification","dataset":"CIFAR-10, 1000 Labels","model":"Dual Student (600)","rank_in_archive_order":5,"of":9,"metrics":{"Accuracy":"85.83"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-cifar-12","task":"Semi-Supervised Image Classification","dataset":"CIFAR-10, 2000 Labels","model":"Dual Student (600)","rank_in_archive_order":3,"of":4,"metrics":{"Accuracy":"89.28"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-cifar","task":"Semi-Supervised Image Classification","dataset":"CIFAR-10, 4000 Labels","model":"Dual Student (600)","rank_in_archive_order":39,"of":49,"metrics":{"Percentage error":"8.89"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-2","task":"Semi-Supervised Image Classification","dataset":"ImageNet - 10% labeled data","model":"Dual Student","rank_in_archive_order":46,"of":75,"metrics":{"Top 1 Accuracy":"63.52%","Top 5 Accuracy":"83.58%"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-svhn-1","task":"Semi-Supervised Image Classification","dataset":"SVHN, 250 Labels","model":"Dual Student","rank_in_archive_order":10,"of":15,"metrics":{"Accuracy":"95.76"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-svhn-3","task":"Semi-Supervised Image Classification","dataset":"SVHN, 500 Labels","model":"Dual Student","rank_in_archive_order":4,"of":6,"metrics":{"Accuracy":"96.04"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-cifar-2","task":"Semi-Supervised Image Classification","dataset":"cifar-100, 10000 Labels","model":"Dual Student (480)","rank_in_archive_order":25,"of":29,"metrics":{"Percentage error":"32.77"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1909.01804","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}