{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evaluating-reliability-in-medical-dnns-a","title":"Evaluating Reliability in Medical DNNs: A Critical Analysis of Feature and Confidence-Based OOD Detection","arxiv_id":"2408.17337","date":"2024-08-30","proceeding":null,"authors":["Harry Anthony","Konstantinos Kamnitsas"],"abstract":"Reliable use of deep neural networks (DNNs) for medical image analysis requires methods to identify inputs that differ significantly from the training data, called out-of-distribution (OOD), to prevent erroneous predictions. OOD detection methods can be categorised as either confidence-based (using the model's output layer for OOD detection) or feature-based (not using the output layer). We created two new OOD benchmarks by dividing the D7P (dermatology) and BreastMNIST (ultrasound) datasets into subsets which either contain or don't contain an artefact (rulers or annotations respectively). Models were trained with artefact-free images, and images with the artefacts were used as OOD test sets. For each OOD image, we created a counterfactual by manually removing the artefact via image processing, to assess the artefact's impact on the model's predictions. We show that OOD artefacts can boost a model's softmax confidence in its predictions, due to correlations in training data among other factors. This contradicts the common assumption that OOD artefacts should lead to more uncertain outputs, an assumption on which most confidence-based methods rely. We use this to explain why feature-based methods (e.g. Mahalanobis score) typically have greater OOD detection performance than confidence-based methods (e.g. MCP). However, we also show that feature-based methods typically perform worse at distinguishing between inputs that lead to correct and incorrect predictions (for both OOD and ID data). Following from these insights, we argue that a combination of feature-based and confidence-based methods should be used within DNN pipelines to mitigate their respective weaknesses. These project's code and OOD benchmarks are available at: https://github.com/HarryAnthony/Evaluating_OOD_detection.","url_abs":"https://arxiv.org/abs/2408.17337v1","url_pdf":"https://arxiv.org/pdf/2408.17337v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evaluating-reliability-in-medical-dnns-a","repo_url":"https://github.com/harryanthony/evaluating_ood_detection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"medical-image-analysis","task_name":"Medical Image Analysis"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2408.17337","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.17337"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/harryanthony/evaluating_ood_detection","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5f4acd853c64f1cc","entry":"apply_truncation_hook","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/ReAct.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/ReAct.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f4acd853c64f1cc"}},{"code_sha256_prefix":"6d387f750d46f99b","entry":"calculate_contribution_matrix","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/DICE.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/DICE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d387f750d46f99b"}},{"code_sha256_prefix":"cdf914b7623c6a1b","entry":"calculate_gram_matrices","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/GRAM.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/GRAM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdf914b7623c6a1b"}},{"code_sha256_prefix":"7aef398e6349fd58","entry":"create_masking_matrix","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/DICE.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/DICE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7aef398e6349fd58"}},{"code_sha256_prefix":"5593340ed4984c72","entry":"extract_upper_triangular","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/GRAM.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/GRAM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5593340ed4984c72"}},{"code_sha256_prefix":"bce4f364c59dbb93","entry":"gradnorm","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/gradnorm.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/gradnorm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bce4f364c59dbb93"}},{"code_sha256_prefix":"b263e7f1e434932f","entry":"modify_transforms","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"make_synthetic_artefacts.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/make_synthetic_artefacts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b263e7f1e434932f"}},{"code_sha256_prefix":"879067ef117fce51","entry":"softmax_entropy","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/entropy.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/entropy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"879067ef117fce51"}},{"code_sha256_prefix":"a272a0afd546b8d5","entry":"calculate_kl_divergence","repo":"harryanthony/evaluating_ood_detection","repo_kind":"official","path":"source/methods/gradnorm.py","file_url":"https://github.com/harryanthony/evaluating_ood_detection/blob/HEAD/source/methods/gradnorm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a272a0afd546b8d5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}