{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multimodal-industrial-anomaly-detection-via","title":"Multimodal Industrial Anomaly Detection via Hybrid Fusion","arxiv_id":"2303.00601","date":"2023-03-01","proceeding":"CVPR 2023 1","authors":["Yue Wang","Jinlong Peng","Jiangning Zhang","Ran Yi","Yabiao Wang","Chengjie Wang"],"abstract":"2D-based Industrial Anomaly Detection has been widely discussed, however, multimodal industrial anomaly detection based on 3D point clouds and RGB images still has many untouched fields. Existing multimodal industrial anomaly detection methods directly concatenate the multimodal features, which leads to a strong disturbance between features and harms the detection performance. In this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly detection method with hybrid fusion scheme: firstly, we design an unsupervised feature fusion with patch-wise contrastive learning to encourage the interaction of different modal features; secondly, we use a decision layer fusion with multiple memory banks to avoid loss of information and additional novelty classifiers to make the final decision. We further propose a point feature alignment operation to better align the point cloud and RGB features. Extensive experiments show that our multimodal industrial anomaly detection model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTec-3D AD dataset. Code is available at https://github.com/nomewang/M3DM.","url_abs":"https://arxiv.org/abs/2303.00601v2","url_pdf":"https://arxiv.org/pdf/2303.00601v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multimodal-industrial-anomaly-detection-via","repo_url":"https://github.com/nomewang/m3dm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-anomaly-detection","task_name":"3D Anomaly Detection"},{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"rgb-3d-anomaly-detection-and-segmentation","task_name":"RGB+3D Anomaly Detection and Segmentation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-anomaly-detection-on-anomaly-shapenet10","task":"3D Anomaly Detection","dataset":"Anomaly-ShapeNet10","model":"M3DM","rank_in_archive_order":4,"of":7,"metrics":{"O-AUROC":"0.574","P-AUROC":"0.648"},"uses_additional_data":false},{"leaderboard":"/sota/3d-anomaly-detection-on-real-3d-ad","task":"3D Anomaly Detection","dataset":"Real 3D-AD","model":"M3DM (PointMAE)","rank_in_archive_order":16,"of":19,"metrics":{"Mean Performance of P. and O. ":"0.5945","Object AUROC":"0.552","Point AUROC":"0.637"},"uses_additional_data":false},{"leaderboard":"/sota/3d-anomaly-detection-on-real-3d-ad","task":"3D Anomaly Detection","dataset":"Real 3D-AD","model":"M3DM (PointBERT)","rank_in_archive_order":18,"of":19,"metrics":{"Mean Performance of P. and O. ":"0.587","Object AUROC":"0.538","Point AUROC":"0.636"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-3d-anomaly-detection-and-segmentation-on","task":"RGB+3D Anomaly Detection and Segmentation","dataset":"MVTEC 3D-AD","model":"M3DM","rank_in_archive_order":4,"of":9,"metrics":{"Detection AUCROC":"0.945","Segmentation AUCROC":"0.992","Segmentation AUPRO":"0.964"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.00601","atlas_url":"https://app.syntology.ai/?focus=2303.00601","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.00601"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nomewang/m3dm","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/nomewang/M3DM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bd3578806ecf5ac2","entry":"adjust_learning_rate","repo":"nomewang/M3DM","repo_kind":"official","path":"utils/lr_sched.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/utils/lr_sched.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd3578806ecf5ac2"}},{"code_sha256_prefix":"3dc19396537db789","entry":"all_reduce_mean","repo":"nomewang/M3DM","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3dc19396537db789"}},{"code_sha256_prefix":"ec413739d406e611","entry":"pc_normalize","repo":"nomewang/M3DM","repo_kind":"official","path":"models/pointnet2_utils.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/models/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec413739d406e611"}},{"code_sha256_prefix":"18c9bfa7e94951e9","entry":"collect_anomaly_scores","repo":"nomewang/M3DM","repo_kind":"official","path":"utils/au_pro_util.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/utils/au_pro_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18c9bfa7e94951e9"}},{"code_sha256_prefix":"49b5692178958afe","entry":"compute_pro","repo":"nomewang/M3DM","repo_kind":"official","path":"utils/au_pro_util.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/utils/au_pro_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49b5692178958afe"}},{"code_sha256_prefix":"4d4e430ec99ab841","entry":"get_data_loader","repo":"nomewang/M3DM","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d4e430ec99ab841"}},{"code_sha256_prefix":"6ddec81b1d23c787","entry":"square_distance","repo":"nomewang/M3DM","repo_kind":"official","path":"models/pointnet2_utils.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/models/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ddec81b1d23c787"}},{"code_sha256_prefix":"b1227ddb721e2999","entry":"timeit","repo":"nomewang/M3DM","repo_kind":"official","path":"models/pointnet2_utils.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/models/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1227ddb721e2999"}},{"code_sha256_prefix":"9d72c1f1f0d6dea4","entry":"trapezoid","repo":"nomewang/M3DM","repo_kind":"official","path":"utils/au_pro_util.py","file_url":"https://github.com/nomewang/M3DM/blob/HEAD/utils/au_pro_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d72c1f1f0d6dea4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}