{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/moe-fusion-instance-embedded-mixture-of","title":"Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion","arxiv_id":"2302.01392","date":"2023-02-02","proceeding":"ICCV 2023 1","authors":["Yiming Sun","Bing Cao","Pengfei Zhu","QinGhua Hu"],"abstract":"Infrared and visible image fusion aims to integrate comprehensive information from multiple sources to achieve superior performances on various practical tasks, such as detection, over that of a single modality. However, most existing methods directly combined the texture details and object contrast of different modalities, ignoring the dynamic changes in reality, which diminishes the visible texture in good lighting conditions and the infrared contrast in low lighting conditions. To fill this gap, we propose a dynamic image fusion framework with a multi-modal gated mixture of local-to-global experts, termed MoE-Fusion, to dynamically extract effective and comprehensive information from the respective modalities. Our model consists of a Mixture of Local Experts (MoLE) and a Mixture of Global Experts (MoGE) guided by a multi-modal gate. The MoLE performs specialized learning of multi-modal local features, prompting the fused images to retain the local information in a sample-adaptive manner, while the MoGE focuses on the global information that complements the fused image with overall texture detail and contrast. Extensive experiments show that our MoE-Fusion outperforms state-of-the-art methods in preserving multi-modal image texture and contrast through the local-to-global dynamic learning paradigm, and also achieves superior performance on detection tasks. Our code will be available: https://github.com/SunYM2020/MoE-Fusion.","url_abs":"https://arxiv.org/abs/2302.01392v2","url_pdf":"https://arxiv.org/pdf/2302.01392v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"moe-fusion-instance-embedded-mixture-of","repo_url":"https://github.com/sunym2020/moe-fusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"infrared-and-visible-image-fusion","task_name":"Infrared And Visible Image Fusion"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2302.01392","atlas_url":"https://app.syntology.ai/?focus=2302.01392","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.01392"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunym2020/moe-fusion","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":2,"ran":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"334c71c1848e108a","entry":"cal_line_length","repo":"sunym2020/moe-fusion","repo_kind":"official","path":"DOTA_devkit/ImgSplit.py","file_url":"https://github.com/sunym2020/moe-fusion/blob/HEAD/DOTA_devkit/ImgSplit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"334c71c1848e108a"}},{"code_sha256_prefix":"787051d64e55a326","entry":"choose_best_pointorder_fit_another","repo":"sunym2020/moe-fusion","repo_kind":"official","path":"DOTA_devkit/ImgSplit.py","file_url":"https://github.com/sunym2020/moe-fusion/blob/HEAD/DOTA_devkit/ImgSplit.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"787051d64e55a326"}},{"code_sha256_prefix":"98c4d71247a0078c","entry":"nmsbynamedict","repo":"sunym2020/moe-fusion","repo_kind":"official","path":"DOTA_devkit/ResultMerge.py","file_url":"https://github.com/sunym2020/moe-fusion/blob/HEAD/DOTA_devkit/ResultMerge.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98c4d71247a0078c"}},{"code_sha256_prefix":"5502a789c2da0e5e","entry":"py_cpu_nms","repo":"sunym2020/moe-fusion","repo_kind":"official","path":"DOTA_devkit/ResultMerge.py","file_url":"https://github.com/sunym2020/moe-fusion/blob/HEAD/DOTA_devkit/ResultMerge.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5502a789c2da0e5e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}