{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/region-mutual-information-loss-for-semantic","title":"Region Mutual Information Loss for Semantic Segmentation","arxiv_id":"1910.12037","date":"2019-10-26","proceeding":"NeurIPS 2019 12","authors":["Shuai Zhao","Yang Wang","Zheng Yang","Deng Cai"],"abstract":"Semantic segmentation is a fundamental problem in computer vision. It is considered as a pixel-wise classification problem in practice, and most segmentation models use a pixel-wise loss as their optimization riterion. However, the pixel-wise loss ignores the dependencies between pixels in an image. Several ways to exploit the relationship between pixels have been investigated, \\eg, conditional random fields (CRF) and pixel affinity based methods. Nevertheless, these methods usually require additional model branches, large extra memories, or more inference time. In this paper, we develop a region mutual information (RMI) loss to model the dependencies among pixels more simply and efficiently. In contrast to the pixel-wise loss which treats the pixels as independent samples, RMI uses one pixel and its neighbour pixels to represent this pixel. Then for each pixel in an image, we get a multi-dimensional point that encodes the relationship between pixels, and the image is cast into a multi-dimensional distribution of these high-dimensional points. The prediction and ground truth thus can achieve high order consistency through maximizing the mutual information (MI) between their multi-dimensional distributions. Moreover, as the actual value of the MI is hard to calculate, we derive a lower bound of the MI and maximize the lower bound to maximize the real value of the MI. RMI only requires a few extra computational resources in the training stage, and there is no overhead during testing. Experimental results demonstrate that RMI can achieve substantial and consistent improvements in performance on PASCAL VOC 2012 and CamVid datasets. The code is available at https://github.com/ZJULearning/RMI.","url_abs":"https://arxiv.org/abs/1910.12037v1","url_pdf":"https://arxiv.org/pdf/1910.12037v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"region-mutual-information-loss-for-semantic","repo_url":"https://github.com/ZJULearning/RMI","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"region-mutual-information-loss-for-semantic","repo_url":"https://github.com/RElbers/region-mutual-information-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.12037","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.12037"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/RElbers/region-mutual-information-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZJULearning/RMI","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_violates":2,"ran_draft_wrong":1,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1},"listed":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"daf6b8b196e08963","entry":"RMILoss","repo":"RElbers/region-mutual-information-pytorch","repo_kind":"listed","path":"rmi/rmi.py","file_url":"https://github.com/RElbers/region-mutual-information-pytorch/blob/HEAD/rmi/rmi.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daf6b8b196e08963"}},{"code_sha256_prefix":"c188738b3a4fca95","entry":"inverse","repo":"RElbers/region-mutual-information-pytorch","repo_kind":"listed","path":"rmi/rmi.py","file_url":"https://github.com/RElbers/region-mutual-information-pytorch/blob/HEAD/rmi/rmi.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c188738b3a4fca95"}},{"code_sha256_prefix":"e3473a5576df4936","entry":"log_det","repo":"RElbers/region-mutual-information-pytorch","repo_kind":"listed","path":"rmi/rmi.py","file_url":"https://github.com/RElbers/region-mutual-information-pytorch/blob/HEAD/rmi/rmi.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3473a5576df4936"}},{"code_sha256_prefix":"30b82ca9d0e092a7","entry":"log_trace","repo":"RElbers/region-mutual-information-pytorch","repo_kind":"listed","path":"rmi/rmi.py","file_url":"https://github.com/RElbers/region-mutual-information-pytorch/blob/HEAD/rmi/rmi.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30b82ca9d0e092a7"}},{"code_sha256_prefix":"55f1419d4d8c483c","entry":"transpose","repo":"RElbers/region-mutual-information-pytorch","repo_kind":"listed","path":"rmi/rmi.py","file_url":"https://github.com/RElbers/region-mutual-information-pytorch/blob/HEAD/rmi/rmi.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55f1419d4d8c483c"}},{"code_sha256_prefix":"e3468efc8c8a04e0","entry":"add_parser_params","repo":"ZJULearning/RMI","repo_kind":"official","path":"parser_params.py","file_url":"https://github.com/ZJULearning/RMI/blob/HEAD/parser_params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3468efc8c8a04e0"}},{"code_sha256_prefix":"c48d876b9a3ae27a","entry":"log_det_by_cholesky","repo":"ZJULearning/RMI","repo_kind":"official","path":"losses/rmi/rmi_utils.py","file_url":"https://github.com/ZJULearning/RMI/blob/HEAD/losses/rmi/rmi_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c48d876b9a3ae27a"}},{"code_sha256_prefix":"97f5e3b69f2c69ea","entry":"map_get_pairs","repo":"ZJULearning/RMI","repo_kind":"official","path":"losses/rmi/rmi_utils.py","file_url":"https://github.com/ZJULearning/RMI/blob/HEAD/losses/rmi/rmi_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97f5e3b69f2c69ea"}},{"code_sha256_prefix":"99b42d109e83351a","entry":"map_get_pairs_region","repo":"ZJULearning/RMI","repo_kind":"official","path":"losses/rmi/rmi_utils.py","file_url":"https://github.com/ZJULearning/RMI/blob/HEAD/losses/rmi/rmi_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99b42d109e83351a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}