{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/metaood-automatic-selection-of-ood-detection","title":"MetaOOD: Automatic Selection of OOD Detection Models","arxiv_id":"2410.03074","date":"2024-10-04","proceeding":null,"authors":["Yuehan Qin","Yichi Zhang","Yi Nian","Xueying Ding","Yue Zhao"],"abstract":"How can we automatically select an out-of-distribution (OOD) detection model for various underlying tasks? This is crucial for maintaining the reliability of open-world applications by identifying data distribution shifts, particularly in critical domains such as online transactions, autonomous driving, and real-time patient diagnosis. Despite the availability of numerous OOD detection methods, the challenge of selecting an optimal model for diverse tasks remains largely underexplored, especially in scenarios lacking ground truth labels. In this work, we introduce MetaOOD, the first zero-shot, unsupervised framework that utilizes meta-learning to automatically select an OOD detection model. As a meta-learning approach, MetaOOD leverages historical performance data of existing methods across various benchmark OOD datasets, enabling the effective selection of a suitable model for new datasets without the need for labeled data at the test time. To quantify task similarities more accurately, we introduce language model-based embeddings that capture the distinctive OOD characteristics of both datasets and detection models. Through extensive experimentation with 24 unique test dataset pairs to choose from among 11 OOD detection models, we demonstrate that MetaOOD significantly outperforms existing methods and only brings marginal time overhead. Our results, validated by Wilcoxon statistical tests, show that MetaOOD surpasses a diverse group of 11 baselines, including established OOD detectors and advanced unsupervised selection methods.","url_abs":"https://arxiv.org/abs/2410.03074v1","url_pdf":"https://arxiv.org/pdf/2410.03074v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"ood-detection","task_name":"Out of Distribution (OOD) Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.03074","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.03074"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yqin43/metaood","reach":null}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"18dd1f7f9d71a93e","entry":"Utils","repo":"yqin43/metaood","repo_kind":"found_in_text","path":"metaood/metaood_0.py","file_url":"https://github.com/yqin43/metaood/blob/HEAD/metaood/metaood_0.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18dd1f7f9d71a93e"}},{"code_sha256_prefix":"ee6c756d2eb41474","entry":"fit","repo":"yqin43/metaood","repo_kind":"found_in_text","path":"metaood/metaood_0.py","file_url":"https://github.com/yqin43/metaood/blob/HEAD/metaood/metaood_0.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee6c756d2eb41474"}},{"code_sha256_prefix":"d15339a4e99e1a88","entry":"meta_predictor","repo":"yqin43/metaood","repo_kind":"found_in_text","path":"metaood/metaood_0.py","file_url":"https://github.com/yqin43/metaood/blob/HEAD/metaood/metaood_0.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d15339a4e99e1a88"}},{"code_sha256_prefix":"959ebdf279f5e4e0","entry":"run","repo":"yqin43/metaood","repo_kind":"found_in_text","path":"metaood/metaood_0.py","file_url":"https://github.com/yqin43/metaood/blob/HEAD/metaood/metaood_0.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"959ebdf279f5e4e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}