{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multiood-scaling-out-of-distribution","title":"MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities","arxiv_id":"2405.17419","date":"2024-05-27","proceeding":null,"authors":["Hao Dong","Yue Zhao","Eleni Chatzi","Olga Fink"],"abstract":"Detecting out-of-distribution (OOD) samples is important for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. Existing research has mainly focused on unimodal scenarios on image data. However, real-world applications are inherently multimodal, which makes it essential to leverage information from multiple modalities to enhance the efficacy of OOD detection. To establish a foundation for more realistic Multimodal OOD Detection, we introduce the first-of-its-kind benchmark, MultiOOD, characterized by diverse dataset sizes and varying modality combinations. We first evaluate existing unimodal OOD detection algorithms on MultiOOD, observing that the mere inclusion of additional modalities yields substantial improvements. This underscores the importance of utilizing multiple modalities for OOD detection. Based on the observation of Modality Prediction Discrepancy between in-distribution (ID) and OOD data, and its strong correlation with OOD performance, we propose the Agree-to-Disagree (A2D) algorithm to encourage such discrepancy during training. Moreover, we introduce a novel outlier synthesis method, NP-Mix, which explores broader feature spaces by leveraging the information from nearest neighbor classes and complements A2D to strengthen OOD detection performance. Extensive experiments on MultiOOD demonstrate that training with A2D and NP-Mix improves existing OOD detection algorithms by a large margin. Our source code and MultiOOD benchmark are available at https://github.com/donghao51/MultiOOD.","url_abs":"https://arxiv.org/abs/2405.17419v2","url_pdf":"https://arxiv.org/pdf/2405.17419v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multiood-scaling-out-of-distribution","repo_url":"https://github.com/donghao51/multiood","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[],"datasets_introduced":[{"slug":"multiood","name":"MultiOOD","full_name":"Multimodal Out-of-Distribution Detection Benchmark"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.17419","atlas_url":"https://app.syntology.ai/?focus=2405.17419","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.17419"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/donghao51/MultiOOD","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/donghao51/multiood","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran":4,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"2a70efefd5fd3fdf","entry":"normalized_prediction_entropy","repo":"donghao51/MultiOOD","repo_kind":"official","path":"EPIC-rgb-flow/train_video_flow_audio_epic.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/EPIC-rgb-flow/train_video_flow_audio_epic.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a70efefd5fd3fdf"}},{"code_sha256_prefix":"7abffed1d8fb63ec","entry":"NetVLAD","repo":"donghao51/multiood","repo_kind":"official","path":"EPIC-rgb-flow/VGGSound/models/resnet.py","file_url":"https://github.com/donghao51/multiood/blob/HEAD/EPIC-rgb-flow/VGGSound/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7abffed1d8fb63ec"}},{"code_sha256_prefix":"0a54a15777a1feeb","entry":"acc","repo":"donghao51/MultiOOD","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a54a15777a1feeb"}},{"code_sha256_prefix":"c1c055d8355879ae","entry":"compute_all_metrics","repo":"donghao51/MultiOOD","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c1c055d8355879ae"}},{"code_sha256_prefix":"e40c372099f6f97e","entry":"fpr_recall","repo":"donghao51/MultiOOD","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e40c372099f6f97e"}},{"code_sha256_prefix":"e5d31e759577909d","entry":"hellinger_distance","repo":"donghao51/multiood","repo_kind":"official","path":"HMDB-rgb-flow/train_video_flow.py","file_url":"https://github.com/donghao51/multiood/blob/HEAD/HMDB-rgb-flow/train_video_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5d31e759577909d"}},{"code_sha256_prefix":"5710019a55c8a04e","entry":"wasserstein_distance","repo":"donghao51/multiood","repo_kind":"official","path":"HMDB-rgb-flow/train_video_flow.py","file_url":"https://github.com/donghao51/multiood/blob/HEAD/HMDB-rgb-flow/train_video_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5710019a55c8a04e"}},{"code_sha256_prefix":"2908b8fe1e90a4e3","entry":"AudioAttGenModule","repo":"donghao51/multiood","repo_kind":"official","path":"EPIC-rgb-flow/VGGSound/models/resnet.py","file_url":"https://github.com/donghao51/multiood/blob/HEAD/EPIC-rgb-flow/VGGSound/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2908b8fe1e90a4e3"}},{"code_sha256_prefix":"c157c58abd3d1ba1","entry":"generalized_entropy","repo":"donghao51/MultiOOD","repo_kind":"official","path":"eval_video_flow_far_ood.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/eval_video_flow_far_ood.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c157c58abd3d1ba1"}},{"code_sha256_prefix":"aa717d479c568823","entry":"get_spectrogram_piece","repo":"donghao51/MultiOOD","repo_kind":"official","path":"HMDB-rgb-flow/dataloader_video_flow_audio.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/HMDB-rgb-flow/dataloader_video_flow_audio.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa717d479c568823"}},{"code_sha256_prefix":"66bf4c7492649567","entry":"load_txt_file","repo":"donghao51/MultiOOD","repo_kind":"official","path":"HMDB-rgb-flow/dataloader_video_flow.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/HMDB-rgb-flow/dataloader_video_flow.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66bf4c7492649567"}},{"code_sha256_prefix":"888f281566803e07","entry":"load_txt_file_kinetics","repo":"donghao51/MultiOOD","repo_kind":"official","path":"HMDB-rgb-flow/dataloader_video_flow.py","file_url":"https://github.com/donghao51/MultiOOD/blob/HEAD/HMDB-rgb-flow/dataloader_video_flow.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"888f281566803e07"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}