{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/l_dmi-an-information-theoretic-noise-robust","title":"L_DMI: An Information-theoretic Noise-robust Loss Function","arxiv_id":"1909.03388","date":"2019-09-08","proceeding":null,"authors":["Yilun Xu","Peng Cao","Yuqing Kong","Yizhou Wang"],"abstract":"Accurately annotating large scale dataset is notoriously expensive both in time and in money. Although acquiring low-quality-annotated dataset can be much cheaper, it often badly damages the performance of trained models when using such dataset without particular treatment. Various methods have been proposed for learning with noisy labels. However, most methods only handle limited kinds of noise patterns, require auxiliary information or steps (e.g. , knowing or estimating the noise transition matrix), or lack theoretical justification. In this paper, we propose a novel information-theoretic loss function, $\\mathcal{L}_{DMI}$, for training deep neural networks robust to label noise. The core of $\\mathcal{L}_{DMI}$ is a generalized version of mutual information, termed Determinant based Mutual Information (DMI), which is not only information-monotone but also relatively invariant. \\emph{To the best of our knowledge, $\\mathcal{L}_{DMI}$ is the first loss function that is provably robust to instance-independent label noise, regardless of noise pattern, and it can be applied to any existing classification neural networks straightforwardly without any auxiliary information}. In addition to theoretical justification, we also empirically show that using $\\mathcal{L}_{DMI}$ outperforms all other counterparts in the classification task on both image dataset and natural language dataset include Fashion-MNIST, CIFAR-10, Dogs vs. Cats, MR with a variety of synthesized noise patterns and noise amounts, as well as a real-world dataset Clothing1M. Codes are available at https://github.com/Newbeeer/L_DMI .","url_abs":"https://arxiv.org/abs/1909.03388v2","url_pdf":"https://arxiv.org/pdf/1909.03388v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"l_dmi-an-information-theoretic-noise-robust","repo_url":"https://github.com/Newbeeer/L_DMI","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"l_dmi-an-information-theoretic-noise-robust","repo_url":"https://github.com/kumar10725/Keras-Determinant-based-Mutual-Information-loss-DMI-loss-","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Unlicense"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"learning-with-noisy-labels","task_name":"Learning with noisy labels"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-clothing1m","task":"Image Classification","dataset":"Clothing1M","model":"DMI","rank_in_archive_order":38,"of":51,"metrics":{"Accuracy":"72.46%"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1909.03388","atlas_url":"https://app.syntology.ai/?focus=1909.03388","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.03388"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Newbeeer/L_DMI","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kumar10725/Keras-Determinant-based-Mutual-Information-loss-DMI-loss-","reach":{"status":"ok","spdx":"Unlicense"}}],"summary":{"ran_draft_wrong":1,"ran_violates":1,"ran_honours":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"dc221e9b2bf38877","entry":"get_int","repo":"Newbeeer/L_DMI","repo_kind":"official","path":"fashion/fashion.py","file_url":"https://github.com/Newbeeer/L_DMI/blob/HEAD/fashion/fashion.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc221e9b2bf38877"}},{"code_sha256_prefix":"10bd51622c10d0ff","entry":"parse_byte","repo":"Newbeeer/L_DMI","repo_kind":"official","path":"fashion/fashion.py","file_url":"https://github.com/Newbeeer/L_DMI/blob/HEAD/fashion/fashion.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"10bd51622c10d0ff"}},{"code_sha256_prefix":"d2b1ee48c5fdcfd2","entry":"read_label_file","repo":"Newbeeer/L_DMI","repo_kind":"official","path":"fashion/fashion.py","file_url":"https://github.com/Newbeeer/L_DMI/blob/HEAD/fashion/fashion.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2b1ee48c5fdcfd2"}},{"code_sha256_prefix":"e02ca4e69c4e0b86","entry":"validate","repo":"Newbeeer/L_DMI","repo_kind":"official","path":"CIFAR-10/DMI.py","file_url":"https://github.com/Newbeeer/L_DMI/blob/HEAD/CIFAR-10/DMI.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e02ca4e69c4e0b86"}},{"code_sha256_prefix":"f11fc12be30eb611","entry":"DMI_loss","repo":"Newbeeer/L_DMI","repo_kind":"official","path":"CIFAR-10/DMI.py","file_url":"https://github.com/Newbeeer/L_DMI/blob/HEAD/CIFAR-10/DMI.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f11fc12be30eb611"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}