{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cfmw-cross-modality-fusion-mamba-for","title":"CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather Conditions","arxiv_id":"2404.16302","date":"2024-04-25","proceeding":null,"authors":["Haoyuan Li","Qi Hu","You Yao","Kailun Yang","Peng Chen"],"abstract":"Cross-modality images that integrate visible-infrared spectra cues can provide richer complementary information for object detection. Despite this, existing visible-infrared object detection methods severely degrade in severe weather conditions. This failure stems from the pronounced sensitivity of visible images to environmental perturbations, such as rain, haze, and snow, which frequently cause false negatives and false positives in detection. To address this issue, we introduce a novel and challenging task, termed visible-infrared object detection under adverse weather conditions. To foster this task, we have constructed a new Severe Weather Visible-Infrared Dataset (SWVID) with diverse severe weather scenes. Furthermore, we introduce the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment detection accuracy in adverse weather conditions. Thanks to the proposed Weather Removal Diffusion Model (WRDM) and Cross-modality Fusion Mamba (CFM) modules, CFMW is able to mine more essential information of pedestrian features in cross-modality fusion, thus could transfer to other rarer scenarios with high efficiency and has adequate availability on those platforms with low computing power. To the best of our knowledge, this is the first study that targeted improvement and integrated both Diffusion and Mamba modules in cross-modality object detection, successfully expanding the practical application of this type of model with its higher accuracy and more advanced architecture. Extensive experiments on both well-recognized and self-created datasets conclusively demonstrate that our CFMW achieves state-of-the-art detection performance, surpassing existing benchmarks. The dataset and source code will be made publicly available at https://github.com/lhy-zjut/CFMW.","url_abs":"https://arxiv.org/abs/2404.16302v1","url_pdf":"https://arxiv.org/pdf/2404.16302v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cfmw-cross-modality-fusion-mamba-for","repo_url":"https://github.com/lhy-zjut/cfmw","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"AGPL-3.0"}}],"tasks":[{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"multispectral-object-detection","task_name":"Multispectral Object Detection"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.16302","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.16302"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lhy-zjut/cfmw","reach":{"status":"ok","spdx":"AGPL-3.0"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"ran_fixture":2,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"988a3c854b1b13d0","entry":"autopad","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/models/common.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"988a3c854b1b13d0"}},{"code_sha256_prefix":"bd1f17e427bf51a5","entry":"dict2namespace","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"PADM/eval_diffusion.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/PADM/eval_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"bd1f17e427bf51a5"}},{"code_sha256_prefix":"82a15cc1e46f7e4d","entry":"to_3d","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"vim/mamba_module.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/vim/mamba_module.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"82a15cc1e46f7e4d"}},{"code_sha256_prefix":"b20f2a5df739a59e","entry":"to_4d","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"vim/mamba_module.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/vim/mamba_module.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b20f2a5df739a59e"}},{"code_sha256_prefix":"b9a38c519e71eeba","entry":"DWConv","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/models/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b9a38c519e71eeba"}},{"code_sha256_prefix":"7f938e0da00ca908","entry":"get_value","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"global_var.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/global_var.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7f938e0da00ca908"}},{"code_sha256_prefix":"386a04c4b6c96c88","entry":"smooth_BCE","repo":"lhy-zjut/cfmw","repo_kind":"official","path":"utils/loss.py","file_url":"https://github.com/lhy-zjut/cfmw/blob/HEAD/utils/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"386a04c4b6c96c88"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}