{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/typicalness-aware-learning-for-failure","title":"Typicalness-Aware Learning for Failure Detection","arxiv_id":"2411.01981","date":"2024-11-04","proceeding":null,"authors":["Yijun Liu","Jiequan Cui","Zhuotao Tian","Senqiao Yang","Qingdong He","Xiaoling Wang","Jingyong Su"],"abstract":"Deep neural networks (DNNs) often suffer from the overconfidence issue, where incorrect predictions are made with high confidence scores, hindering the applications in critical systems. In this paper, we propose a novel approach called Typicalness-Aware Learning (TAL) to address this issue and improve failure detection performance. We observe that, with the cross-entropy loss, model predictions are optimized to align with the corresponding labels via increasing logit magnitude or refining logit direction. However, regarding atypical samples, the image content and their labels may exhibit disparities. This discrepancy can lead to overfitting on atypical samples, ultimately resulting in the overconfidence issue that we aim to address. To tackle the problem, we have devised a metric that quantifies the typicalness of each sample, enabling the dynamic adjustment of the logit magnitude during the training process. By allowing atypical samples to be adequately fitted while preserving reliable logit direction, the problem of overconfidence can be mitigated. TAL has been extensively evaluated on benchmark datasets, and the results demonstrate its superiority over existing failure detection methods. Specifically, TAL achieves a more than 5% improvement on CIFAR100 in terms of the Area Under the Risk-Coverage Curve (AURC) compared to the state-of-the-art. Code is available at https://github.com/liuyijungoon/TAL.","url_abs":"https://arxiv.org/abs/2411.01981v2","url_pdf":"https://arxiv.org/pdf/2411.01981v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"typicalness-aware-learning-for-failure","repo_url":"https://github.com/liuyijungoon/tal","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.01981","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.01981"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/liuyijungoon/TAL","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liuyijungoon/tal","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"ran":1,"ran_draft_wrong":2,"ran_fixture":2,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3fb68ed6809a8085","entry":"sigmoid_rampup","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fb68ed6809a8085"}},{"code_sha256_prefix":"e2ff478922f1aa7e","entry":"AverageMeter","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2ff478922f1aa7e"}},{"code_sha256_prefix":"a7690d61376264fc","entry":"ConvMixer","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/convmixer.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/convmixer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7690d61376264fc"}},{"code_sha256_prefix":"67a13c0cb19afb83","entry":"accuracy","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67a13c0cb19afb83"}},{"code_sha256_prefix":"6af95ebe99af2e36","entry":"conv3x3","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/resnet.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/resnet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6af95ebe99af2e36"}},{"code_sha256_prefix":"93b5d8a8dc36d1ca","entry":"get_current_consistency_weight","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93b5d8a8dc36d1ca"}},{"code_sha256_prefix":"225bd91e98982a6f","entry":"pca_torch","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"225bd91e98982a6f"}},{"code_sha256_prefix":"f72c08537f7bcb61","entry":"ResNet18","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/resnet18.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/resnet18.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f72c08537f7bcb61"}},{"code_sha256_prefix":"fe6282e5693bcf7f","entry":"ResNet50","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/resnet18.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/resnet18.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe6282e5693bcf7f"}},{"code_sha256_prefix":"4fa7bc5b18c28b84","entry":"efficientnet","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/efficientnet.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/efficientnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4fa7bc5b18c28b84"}},{"code_sha256_prefix":"9b3e549e56ca87bb","entry":"mobilenet","repo":"liuyijungoon/TAL","repo_kind":"official","path":"model/mobilenet.py","file_url":"https://github.com/liuyijungoon/TAL/blob/HEAD/model/mobilenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b3e549e56ca87bb"}},{"code_sha256_prefix":"1c7f6c7e81a98693","entry":"train","repo":"liuyijungoon/tal","repo_kind":"official","path":"trainer/train_TAL.py","file_url":"https://github.com/liuyijungoon/tal/blob/HEAD/trainer/train_TAL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c7f6c7e81a98693"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}