{"url":"/dataset/coco-wan-medium-noise","name":"COCO-WAN (Medium noise)","full_name":"Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters","description_markdown":"The COCO-WAN benchmark is designed to assess the impact of weakly annotations (combined with auto-annotation tools) noise on instance segmentation models. This benchmark is built upon the COCO dataset and incorporates noise generated through weak annotations, simulating real-world scenarios where annotations might be imperfect due to semi-automated tools. It includes various levels of noise to challenge the robustness and generalization capabilities of segmentation models.\r\n\r\nAccurately labeling instance segmentation datasets is a complex and error-prone task, often leading to noisy labels. The COCO-WAN benchmark aims to provide a realistic testing ground for models to handle such noisy annotations. By utilizing foundation models and weak annotations, COCO-WAN simulates semi-automated annotation tools, helping researchers understand how well their models can perform under less-than-ideal labeling conditions. This benchmark includes multiple noise levels (easy, medium, and hard) to reflect varying degrees of annotation imperfections.\r\n\r\nPotential Use Cases of the Dataset:\r\n\r\nModel Robustness Testing: Researchers can use COCO-WAN to evaluate how different instance segmentation models cope with noisy annotations, allowing for the development of more resilient algorithms.\r\nAnnotation Tool Improvement: By analyzing model performance on COCO-WAN, developers of annotation tools can identify common pitfalls and work on reducing noise in their outputs.\r\n\r\nSemi-Automated Annotation Systems: The benchmark provides insights into how models trained with semi-automated annotations perform, guiding improvements in such systems for better accuracy and efficiency in labeling tasks.\r\nThe COCO-WAN benchmark offers a crucial resource for advancing the field of instance segmentation by highlighting the challenges posed by noisy labels and fostering the creation of more robust and reliable models.\r\n\r\nModel Robustness Testing: Researchers can use COCO-WAN to evaluate how different instance segmentation models cope with spatial, real noisy annotations, allowing for the development of more resilient algorithms.\r\n\r\nSemi-Automated Annotation Systems: The benchmark provides insights into how models trained with semi-automated annotations perform, guiding improvements in such systems for better accuracy and efficiency in labeling tasks.\r\n\r\nThe COCO-WAN benchmark offers a crucial resource for advancing the field of instance segmentation by highlighting the challenges posed by noisy labels and fostering the creation of more robust and reliable models.","description_withheld":null,"homepage":"https://github.com/eden500/Noisy-Labels-Instance-Segmentation","introduced_date":"2024-06-16","introduced_date_note":null,"introduced_by":{"paper":"/paper/benchmarking-label-noise-in-instance","title":"Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters","first_author":"Eden Grad","url":null},"license":{"name":"CC by 4.0","url":"https://cocodataset.org/#termsofuse"},"modalities":[{"name":"Images","url":"/datasets/modality/images"}],"tasks":[{"name":"Instance Segmentation","url":"/task/instance-segmentation","datasets_with_task":"/datasets/task/instance-segmentation"},{"name":"Learning with noisy labels","url":"/task/learning-with-noisy-labels","datasets_with_task":"/datasets/task/learning-with-noisy-labels"}],"languages":[],"variants":["COCO-WAN","COCO-WAN (Medium noise)"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/learning-with-noisy-labels-on-coco-wan","task":"Learning with noisy labels","dataset_variant":"COCO-WAN","rows":1,"metrics":["mIOU"],"first_row_in_archive_order":{"model":"Mask R-CNN (ResNet-50-FPN)","paper":"/paper/benchmarking-label-noise-in-instance","metrics":{"mIOU":"25.5"},"code_links":[{"title":"eden500/Noisy-Labels-Instance-Segmentation","url":"https://github.com/eden500/Noisy-Labels-Instance-Segmentation"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/benchmarking-label-noise-in-instance","title":"Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters","date":"2024-06-16","rows_on_this_dataset":1,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":2,"samples_ran":2,"samples_unverified":0,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":1,"samples_harvested":2,"samples_ran":2,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}