{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nnu-net-revisited-a-call-for-rigorous","title":"nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation","arxiv_id":"2404.09556","date":"2024-04-15","proceeding":null,"authors":["Fabian Isensee","Tassilo Wald","Constantin Ulrich","Michael Baumgartner","Saikat Roy","Klaus Maier-Hein","Paul F. Jaeger"],"abstract":"The release of nnU-Net marked a paradigm shift in 3D medical image segmentation, demonstrating that a properly configured U-Net architecture could still achieve state-of-the-art results. Despite this, the pursuit of novel architectures, and the respective claims of superior performance over the U-Net baseline, continued. In this study, we demonstrate that many of these recent claims fail to hold up when scrutinized for common validation shortcomings, such as the use of inadequate baselines, insufficient datasets, and neglected computational resources. By meticulously avoiding these pitfalls, we conduct a thorough and comprehensive benchmarking of current segmentation methods including CNN-based, Transformer-based, and Mamba-based approaches. In contrast to current beliefs, we find that the recipe for state-of-the-art performance is 1) employing CNN-based U-Net models, including ResNet and ConvNeXt variants, 2) using the nnU-Net framework, and 3) scaling models to modern hardware resources. These results indicate an ongoing innovation bias towards novel architectures in the field and underscore the need for more stringent validation standards in the quest for scientific progress.","url_abs":"https://arxiv.org/abs/2404.09556v2","url_pdf":"https://arxiv.org/pdf/2404.09556v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"nnu-net-revisited-a-call-for-rigorous","repo_url":"https://github.com/MIC-DKFZ/nnunet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"nnu-net-revisited-a-call-for-rigorous","repo_url":"https://github.com/DeepMicroscopy/nnUNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"nnu-net-revisited-a-call-for-rigorous","repo_url":"https://github.com/tawald/nnunet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convnext","method_name":"ConvNeXt"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2404.09556","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.09556"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MIC-DKFZ/nnunet","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DeepMicroscopy/nnUNet","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tawald/nnunet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1},"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f8636c471203bf8b","entry":"comp_blosc2_params","repo":"MIC-DKFZ/nnunet","repo_kind":"official","path":"nnunetv2/training/dataloading/nnunet_dataset.py","file_url":"https://github.com/MIC-DKFZ/nnunet/blob/HEAD/nnunetv2/training/dataloading/nnunet_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8636c471203bf8b"}},{"code_sha256_prefix":"58d38fedb5114f60","entry":"calculate_confusion_matrix","repo":"DeepMicroscopy/nnUNet","repo_kind":"listed","path":"nnunetv2/evaluation.py","file_url":"https://github.com/DeepMicroscopy/nnUNet/blob/HEAD/nnunetv2/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"58d38fedb5114f60"}},{"code_sha256_prefix":"a2c8b38ba9393780","entry":"convert_labels_back_to_BraTS","repo":"DeepMicroscopy/nnUNet","repo_kind":"listed","path":"nnunetv2/dataset_conversion/Dataset042_BraTS18.py","file_url":"https://github.com/DeepMicroscopy/nnUNet/blob/HEAD/nnunetv2/dataset_conversion/Dataset042_BraTS18.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2c8b38ba9393780"}},{"code_sha256_prefix":"517ad51ee69e8a2d","entry":"copy_files","repo":"MIC-DKFZ/nnunet","repo_kind":"official","path":"nnunetv2/dataset_conversion/Dataset027_ACDC.py","file_url":"https://github.com/MIC-DKFZ/nnunet/blob/HEAD/nnunetv2/dataset_conversion/Dataset027_ACDC.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"517ad51ee69e8a2d"}},{"code_sha256_prefix":"9235f33692fd804d","entry":"mean_iou","repo":"DeepMicroscopy/nnUNet","repo_kind":"listed","path":"nnunetv2/evaluation.py","file_url":"https://github.com/DeepMicroscopy/nnUNet/blob/HEAD/nnunetv2/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9235f33692fd804d"}},{"code_sha256_prefix":"1ce6a0bc9d7ba540","entry":"mean_iou_from_confusion_matrix","repo":"DeepMicroscopy/nnUNet","repo_kind":"listed","path":"nnunetv2/evaluation.py","file_url":"https://github.com/DeepMicroscopy/nnUNet/blob/HEAD/nnunetv2/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ce6a0bc9d7ba540"}},{"code_sha256_prefix":"86ec2ccc2984b8bf","entry":"merge","repo":"MIC-DKFZ/nnunet","repo_kind":"official","path":"nnunetv2/batch_running/generate_lsf_runs_customDecathlon.py","file_url":"https://github.com/MIC-DKFZ/nnunet/blob/HEAD/nnunetv2/batch_running/generate_lsf_runs_customDecathlon.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"86ec2ccc2984b8bf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}