{"url":"/method/noisy-student","slug":"noisy-student","name":"Noisy Student","full_name":"Noisy Student","full_name_withheld":false,"description_markdown":"**Noisy Student Training** is a semi-supervised learning approach. It extends the idea of self-training\r\nand distillation with the use of equal-or-larger student models and noise added to the student during learning. It has three main steps: \r\n\r\n1. train a teacher model on labeled images\r\n2. use the teacher to generate pseudo labels on unlabeled images\r\n3. train a student model on the combination of labeled images and pseudo labeled images. \r\n\r\nThe algorithm is iterated a few times by treating the student as a teacher to relabel the unlabeled data and training a new student.\r\n\r\nNoisy Student Training seeks to improve on self-training and distillation in two ways. First, it makes the student larger than, or at least equal to, the teacher so the student can better learn from a larger dataset. Second, it adds noise to the student so the noised student is forced to learn harder from the pseudo labels. To noise the student, it uses input noise such as [RandAugment](https://paperswithcode.com/method/randaugment) data augmentation, and model noise such as [dropout](https://paperswithcode.com/method/dropout) and [stochastic depth](https://paperswithcode.com/method/stochastic-depth) during training.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1911.04252v4","title":"Self-training with Noisy Student improves ImageNet classification","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/google-research/noisystudent","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Semi-Supervised Learning Methods","url":"/methods/category/semi-supervised-learning-methods","pwc_aliases":[]}],"n_papers_tagged":38,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/sst-self-training-with-self-adaptive-1","title":"SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning","date":"2025-05-31","arxiv_id":"2506.00467","n_code_links":0,"syntology":null},{"paper":"/paper/zipa-a-family-of-efficient-models-for","title":"ZIPA: A family of efficient models for multilingual phone recognition","date":"2025-05-29","arxiv_id":"2505.23170","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}},{"paper":null,"title":"HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation","date":"2025-04-14","arxiv_id":"2504.09876","n_code_links":0,"syntology":null},{"paper":null,"title":"Improving noisy student training for low-resource languages in End-to-End ASR using CycleGAN and inter-domain losses","date":"2024-07-26","arxiv_id":"2407.21061","n_code_links":0,"syntology":null},{"paper":null,"title":"Semi-supervised Learning for Code-Switching ASR with Large Language Model Filter","date":"2024-07-05","arxiv_id":"2407.04219","n_code_links":0,"syntology":null},{"paper":"/paper/gigaspeech-2-an-evolving-large-scale-and","title":"GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement","date":"2024-06-17","arxiv_id":"2406.11546","n_code_links":2,"syntology":{"ran":1,"of":2,"unverified":1,"pointer_only":0}},{"paper":"/paper/self-train-before-you-transcribe","title":"Self-Train Before You Transcribe","date":"2024-06-17","arxiv_id":"2406.12937","n_code_links":1,"syntology":null},{"paper":null,"title":"Conformer-1: Robust ASR via Large-Scale Semisupervised Bootstrapping","date":"2024-04-10","arxiv_id":"2404.07341","n_code_links":0,"syntology":null},{"paper":null,"title":"On the Effect of Image Resolution on Semantic Segmentation","date":"2024-02-08","arxiv_id":"2402.05398","n_code_links":0,"syntology":null},{"paper":null,"title":"Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR","date":"2024-01-17","arxiv_id":"2401.08992","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning","date":"2024-01-03","arxiv_id":"2401.01473","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-Transriber: Few-shot Lyrics Transcription with Self-training","date":"2022-11-18","arxiv_id":"2211.10152","n_code_links":0,"syntology":null},{"paper":null,"title":"Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR","date":"2022-10-11","arxiv_id":"2210.05793","n_code_links":0,"syntology":null},{"paper":"/paper/progress-and-limitations-of-deep-networks-to","title":"Progress and limitations of deep networks to recognize objects in unusual poses","date":"2022-07-16","arxiv_id":"2207.08034","n_code_links":1,"syntology":null},{"paper":null,"title":"FedNST: Federated Noisy Student Training for Automatic Speech Recognition","date":"2022-06-06","arxiv_id":"2206.02797","n_code_links":0,"syntology":null},{"paper":"/paper/conformer-and-blind-noisy-students-for","title":"Conformer and Blind Noisy Students for Improved Image Quality Assessment","date":"2022-04-27","arxiv_id":"2204.12819","n_code_links":1,"syntology":null},{"paper":"/paper/pseudo-label-transfer-from-frame-level-to","title":"Pseudo-Label Transfer from Frame-Level to Note-Level in a Teacher-Student Framework for Singing Transcription from Polyphonic Music","date":"2022-03-25","arxiv_id":"2203.13422","n_code_links":1,"syntology":null},{"paper":null,"title":"Pseudo Label Is Better Than Human Label","date":"2022-03-22","arxiv_id":"2203.12668","n_code_links":0,"syntology":null},{"paper":"/paper/occlusion-aware-self-supervised-monocular-6d","title":"Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation","date":"2022-03-19","arxiv_id":"2203.10339","n_code_links":1,"syntology":null},{"paper":"/paper/semi-supervised-music-emotion-recognition","title":"Semi-supervised music emotion recognition using noisy student training and harmonic pitch class profiles","date":"2021-12-01","arxiv_id":"2112.00702","n_code_links":1,"syntology":null},{"paper":null,"title":"Using Contrastive Learning and Pseudolabels to learn representations for Retail Product Image Classification","date":"2021-10-07","arxiv_id":"2110.03639","n_code_links":0,"syntology":null},{"paper":"/paper/perturb-predict-paraphrase-semi-supervised","title":"Perturb, Predict & Paraphrase: Semi-Supervised Learning using Noisy Student for Image Captioning","date":"2021-08-19","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification","date":"2021-08-18","arxiv_id":"2108.08362","n_code_links":0,"syntology":null},{"paper":null,"title":"CASPIANET++: A Multidimensional Channel-Spatial Asymmetric Attention Network with Noisy Student Curriculum Learning Paradigm for Brain Tumor Segmentation","date":"2021-07-08","arxiv_id":"2107.04099","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4","date":"2021-07-06","arxiv_id":"2107.02569","n_code_links":0,"syntology":null},{"paper":null,"title":"Semi-supervised Learning for Dense Object Detection in Retail Scenes","date":"2021-07-05","arxiv_id":"2107.02114","n_code_links":0,"syntology":null},{"paper":null,"title":"Predicting Students’ Difficulties From a Piece of Code","date":"2021-06-03","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Deep learning for detecting pulmonary tuberculosis via chest radiography: an international study across 10 countries","date":"2021-05-16","arxiv_id":"2105.07540","n_code_links":0,"syntology":null},{"paper":null,"title":"Semi-supervised learning for generalizable intracranial hemorrhage detection and segmentation","date":"2021-05-03","arxiv_id":"2105.00582","n_code_links":0,"syntology":null},{"paper":null,"title":"A new semi-supervised self-training method for lung cancer prediction","date":"2020-12-17","arxiv_id":"2012.09472","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/speech-recognition","name":"Speech Recognition","papers":10},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":10},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":8},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":7},{"task":"/task/image-classification","name":"Image Classification","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":6},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":4},{"task":"/task/pseudo-label","name":"Pseudo Label","papers":4},{"task":"/task/segmentation","name":"Segmentation","papers":4},{"task":"/task/knowledge-distillation","name":"Knowledge Distillation","papers":3},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":3},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":3},{"task":"/task/classification-1","name":"Classification","papers":2},{"task":"/task/computed-tomography-ct","name":"Computed Tomography (CT)","papers":2},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":2},{"task":"/task/classification","name":"General Classification","papers":2},{"task":"/task/language-modelling","name":"Language Modelling","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/6d-pose-estimation-1","name":"6D Pose Estimation","papers":1},{"task":"/task/6d-pose-estimation","name":"6D Pose Estimation using RGB","papers":1}],"tasks_shown":20,"n_tasks":65,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":8},{"year":"2021","papers":10},{"year":"2022","papers":8},{"year":"2024","papers":8},{"year":"2025","papers":3}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/noisy-student"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}