{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-role-of-label-noise-in-the-feature","title":"On the Role of Label Noise in the Feature Learning Process","arxiv_id":"2505.18909","date":"2025-05-25","proceeding":null,"authors":["Andi Han","Wei Huang","Zhanpeng Zhou","Gang Niu","Wuyang Chen","Junchi Yan","Akiko Takeda","Taiji Suzuki"],"abstract":"Deep learning with noisy labels presents significant challenges. In this work, we theoretically characterize the role of label noise from a feature learning perspective. Specifically, we consider a signal-noise data distribution, where each sample comprises a label-dependent signal and label-independent noise, and rigorously analyze the training dynamics of a two-layer convolutional neural network under this data setup, along with the presence of label noise. Our analysis identifies two key stages. In Stage I, the model perfectly fits all the clean samples (i.e., samples without label noise) while ignoring the noisy ones (i.e., samples with noisy labels). During this stage, the model learns the signal from the clean samples, which generalizes well on unseen data. In Stage II, as the training loss converges, the gradient in the direction of noise surpasses that of the signal, leading to overfitting on noisy samples. Eventually, the model memorizes the noise present in the noisy samples and degrades its generalization ability. Furthermore, our analysis provides a theoretical basis for two widely used techniques for tackling label noise: early stopping and sample selection. Experiments on both synthetic and real-world setups validate our theory.","url_abs":"https://arxiv.org/abs/2505.18909v1","url_pdf":"https://arxiv.org/pdf/2505.18909v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-role-of-label-noise-in-the-feature","repo_url":"https://github.com/zzp1012/label-noise-theory","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"learning-with-noisy-labels","task_name":"Learning with noisy labels"}],"methods":[{"method_slug":"early-stopping","method_name":"Early Stopping"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.18909","atlas_url":"https://app.syntology.ai/?focus=2505.18909","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.18909"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zzp1012/label-noise-theory","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"edd9c782220bb4e8","entry":"get_logger","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/utils/utils.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"edd9c782220bb4e8"}},{"code_sha256_prefix":"2c6366c32294dadd","entry":"get_module","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/utils/tools.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/utils/tools.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c6366c32294dadd"}},{"code_sha256_prefix":"3e5bd5d47c6cbed4","entry":"interpolate_weights","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/utils/tools.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/utils/tools.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e5bd5d47c6cbed4"}},{"code_sha256_prefix":"ae8aef7688525312","entry":"make_layers","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/model/cifar_vgg.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/model/cifar_vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae8aef7688525312"}},{"code_sha256_prefix":"4aa23562fc1fc1a9","entry":"search_by_suffix","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/utils/tools.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/utils/tools.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4aa23562fc1fc1a9"}},{"code_sha256_prefix":"3e7cb9bdaf2ede2d","entry":"set_device","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/utils/utils.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e7cb9bdaf2ede2d"}},{"code_sha256_prefix":"d1e094d5d233a629","entry":"vgg11","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/model/cifar_vgg.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/model/cifar_vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1e094d5d233a629"}},{"code_sha256_prefix":"174e080f41432e65","entry":"vgg11_bn","repo":"zzp1012/label-noise-theory","repo_kind":"official","path":"src/model/cifar_vgg.py","file_url":"https://github.com/zzp1012/label-noise-theory/blob/HEAD/src/model/cifar_vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"174e080f41432e65"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}