{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sure-survey-recipes-for-building-reliable-and","title":"SURE: SUrvey REcipes for building reliable and robust deep networks","arxiv_id":"2403.00543","date":"2024-03-01","proceeding":"CVPR 2024 1","authors":["Yuting Li","Yingyi Chen","Xuanlong Yu","Dexiong Chen","Xi Shen"],"abstract":"In this paper, we revisit techniques for uncertainty estimation within deep neural networks and consolidate a suite of techniques to enhance their reliability. Our investigation reveals that an integrated application of diverse techniques--spanning model regularization, classifier and optimization--substantially improves the accuracy of uncertainty predictions in image classification tasks. The synergistic effect of these techniques culminates in our novel SURE approach. We rigorously evaluate SURE against the benchmark of failure prediction, a critical testbed for uncertainty estimation efficacy. Our results showcase a consistently better performance than models that individually deploy each technique, across various datasets and model architectures. When applied to real-world challenges, such as data corruption, label noise, and long-tailed class distribution, SURE exhibits remarkable robustness, delivering results that are superior or on par with current state-of-the-art specialized methods. Particularly on Animal-10N and Food-101N for learning with noisy labels, SURE achieves state-of-the-art performance without any task-specific adjustments. This work not only sets a new benchmark for robust uncertainty estimation but also paves the way for its application in diverse, real-world scenarios where reliability is paramount. Our code is available at \\url{https://yutingli0606.github.io/SURE/}.","url_abs":"https://arxiv.org/abs/2403.00543v1","url_pdf":"https://arxiv.org/pdf/2403.00543v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sure-survey-recipes-for-building-reliable-and","repo_url":"https://github.com/YutingLi0606/SURE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"learning-with-noisy-labels","task_name":"Learning with noisy labels"},{"task_slug":"long-tail-learning","task_name":"Long-tail Learning"},{"task_slug":"survey","task_name":"Survey"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-food-101n-1","task":"Image Classification","dataset":"Food-101N","model":"SURE(ResNet-50)","rank_in_archive_order":3,"of":4,"metrics":{"Accuracy":"88.0"},"uses_additional_data":false},{"leaderboard":"/sota/learning-with-noisy-labels-on-animal","task":"Learning with noisy labels","dataset":"ANIMAL","model":"SURE","rank_in_archive_order":2,"of":19,"metrics":{"Accuracy":"89.0","ImageNet Pretrained":"NO","Network":"Vgg19-BN"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-10-lt-r-10","task":"Long-tail Learning","dataset":"CIFAR-10-LT (ρ=10)","model":"SURE(ResNet-32)","rank_in_archive_order":2,"of":50,"metrics":{"Error Rate":"5.04"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-10-lt-r-100","task":"Long-tail Learning","dataset":"CIFAR-10-LT (ρ=100)","model":"SURE(ResNet-32)","rank_in_archive_order":6,"of":28,"metrics":{"Error Rate":"13.07"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-10-lt-r-50","task":"Long-tail Learning","dataset":"CIFAR-10-LT (ρ=50)","model":"SURE(ResNet-32)","rank_in_archive_order":2,"of":8,"metrics":{"Error Rate":"9.78"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-100-lt-r-10","task":"Long-tail Learning","dataset":"CIFAR-100-LT (ρ=10)","model":"SURE(ResNet-32)","rank_in_archive_order":7,"of":31,"metrics":{"Error Rate":"26.76"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-100-lt-r-100","task":"Long-tail Learning","dataset":"CIFAR-100-LT (ρ=100)","model":"SURE(ResNet-32)","rank_in_archive_order":9,"of":66,"metrics":{"Error Rate":"43.66"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-100-lt-r-50","task":"Long-tail Learning","dataset":"CIFAR-100-LT (ρ=50)","model":"SURE(ResNet-32)","rank_in_archive_order":7,"of":25,"metrics":{"Error Rate":"36.87"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.00543","atlas_url":"https://app.syntology.ai/?focus=2403.00543","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.00543"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YutingLi0606/SURE","reach":{"status":"ok"}}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"4a20e753857545ed","entry":"calc_aurc_eaurc","repo":"YutingLi0606/SURE","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a20e753857545ed"}},{"code_sha256_prefix":"6328797fa973a062","entry":"calc_ece","repo":"YutingLi0606/SURE","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6328797fa973a062"}},{"code_sha256_prefix":"8f790e315b526f80","entry":"calc_fpr_aupr","repo":"YutingLi0606/SURE","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f790e315b526f80"}},{"code_sha256_prefix":"f71e6609d7f54a29","entry":"compute_confidence_scores","repo":"YutingLi0606/SURE","repo_kind":"official","path":"train_finetune.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/train_finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f71e6609d7f54a29"}},{"code_sha256_prefix":"ac62432dc5134b0d","entry":"make_layers","repo":"YutingLi0606/SURE","repo_kind":"official","path":"model/vgg.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/model/vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac62432dc5134b0d"}},{"code_sha256_prefix":"d577c874c4871163","entry":"compute_loss","repo":"YutingLi0606/SURE","repo_kind":"official","path":"train_finetune.py","file_url":"https://github.com/YutingLi0606/SURE/blob/HEAD/train_finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d577c874c4871163"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}