{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/modeling-multimodal-aleatoric-uncertainty-in","title":"Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts","arxiv_id":"2212.07328","date":"2022-12-14","proceeding":null,"authors":["Zhitong Gao","Yucong Chen","Chuyu Zhang","Xuming He"],"abstract":"Equipping predicted segmentation with calibrated uncertainty is essential for safety-critical applications. In this work, we focus on capturing the data-inherent uncertainty (aka aleatoric uncertainty) in segmentation, typically when ambiguities exist in input images. Due to the high-dimensional output space and potential multiple modes in segmenting ambiguous images, it remains challenging to predict well-calibrated uncertainty for segmentation. To tackle this problem, we propose a novel mixture of stochastic experts (MoSE) model, where each expert network estimates a distinct mode of the aleatoric uncertainty and a gating network predicts the probabilities of an input image being segmented in those modes. This yields an efficient two-level uncertainty representation. To learn the model, we develop a Wasserstein-like loss that directly minimizes the distribution distance between the MoSE and ground truth annotations. The loss can easily integrate traditional segmentation quality measures and be efficiently optimized via constraint relaxation. We validate our method on the LIDC-IDRI dataset and a modified multimodal Cityscapes dataset. Results demonstrate that our method achieves the state-of-the-art or competitive performance on all metrics.","url_abs":"https://arxiv.org/abs/2212.07328v2","url_pdf":"https://arxiv.org/pdf/2212.07328v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"modeling-multimodal-aleatoric-uncertainty-in","repo_url":"https://github.com/gaozhitong/mose-auseg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.07328","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.07328"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gaozhitong/mose-auseg","reach":null}],"summary":{"ran":4,"ran_honours":1,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1},"community":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"64cc9ace598f9303","entry":"Gating","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"64cc9ace598f9303"}},{"code_sha256_prefix":"f66c73d52b935e54","entry":"cal_metrics_batch","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f66c73d52b935e54"}},{"code_sha256_prefix":"9dfce7cb25c702b3","entry":"get_uniform_prob","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9dfce7cb25c702b3"}},{"code_sha256_prefix":"51d24951f7ebb889","entry":"get_weight","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"51d24951f7ebb889"}},{"code_sha256_prefix":"3f73f92d47dc1956","entry":"iou_dist","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f73f92d47dc1956"}},{"code_sha256_prefix":"dcebf3751afb1a72","entry":"MoSE","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dcebf3751afb1a72"}},{"code_sha256_prefix":"e31e897e9545b529","entry":"Noise_injector","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e31e897e9545b529"}},{"code_sha256_prefix":"b985a16c7e304bc8","entry":"build_model","repo":"IMPLabUniPr/swin2-mose","repo_kind":"community","path":"src/super_res/model.py","file_url":"https://github.com/IMPLabUniPr/swin2-mose/blob/HEAD/src/super_res/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b985a16c7e304bc8"}},{"code_sha256_prefix":"9c5ac86b50c73a3f","entry":"get_cost_matrix","repo":"gaozhitong/mose-auseg","repo_kind":"official","path":"models/MoSE.py","file_url":"https://github.com/gaozhitong/mose-auseg/blob/HEAD/models/MoSE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c5ac86b50c73a3f"}},{"code_sha256_prefix":"5574012ba4af04ff","entry":"nanmean","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"5574012ba4af04ff"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}