{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prediction-error-meta-classification-in","title":"Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities","arxiv_id":"1811.00648","date":"2018-11-01","proceeding":null,"authors":["Matthias Rottmann","Pascal Colling","Thomas-Paul Hack","Robin Chan","Fabian Hüger","Peter Schlicht","Hanno Gottschalk"],"abstract":"We present a method that \"meta\" classifies whether seg-ments predicted by a semantic segmentation neural networkintersect with the ground truth. For this purpose, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene's size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. This procedure yields an almost plug and play post-processing tool to rate the prediction quality of semantic segmentation networks on segment level. This is especially relevant for monitoring neural networks in online applications like automated driving or medical imaging where reliability is of utmost importance. In our tests, we use publicly available state-of-the-art networks trained on the Cityscapes dataset and the BraTS2017 dataset and analyze the predictive power of different metrics as well as different sets of metrics. To this end, we compute logistic LASSO regression fits for the task of classifying IoU=0 vs. IoU>0 per segment and obtain AUROC values of up to 91.55%. We complement these tests with linear regression fits to predict the segment-wise IoU and obtain prediction standard deviations of down to 0.130 as well as $R^2$ values of up to 84.15%. We show that these results clearly outperform standard approaches.","url_abs":"https://arxiv.org/abs/1811.00648v2","url_pdf":"https://arxiv.org/pdf/1811.00648v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prediction-error-meta-classification-in","repo_url":"https://github.com/mrottmann/MetaSeg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"linear-regression","method_name":"Linear Regression"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.00648","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.00648"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mrottmann/MetaSeg","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1710ba94506a25eb","entry":"classes_to_categorical","repo":"mrottmann/MetaSeg","repo_kind":"official","path":"metaseg_eval.py","file_url":"https://github.com/mrottmann/MetaSeg/blob/HEAD/metaseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1710ba94506a25eb"}},{"code_sha256_prefix":"a81b247f72776b4c","entry":"label_as_onehot","repo":"mrottmann/MetaSeg","repo_kind":"official","path":"metaseg_eval.py","file_url":"https://github.com/mrottmann/MetaSeg/blob/HEAD/metaseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a81b247f72776b4c"}},{"code_sha256_prefix":"166fe4a3197b4f1a","entry":"visualize_segments","repo":"mrottmann/MetaSeg","repo_kind":"official","path":"metaseg_eval.py","file_url":"https://github.com/mrottmann/MetaSeg/blob/HEAD/metaseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"166fe4a3197b4f1a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}