{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-ood-detectors-in-their-natural","title":"Training OOD Detectors in their Natural Habitats","arxiv_id":"2202.03299","date":"2022-02-07","proceeding":null,"authors":["Julian Katz-Samuels","Julia Nakhleh","Robert Nowak","Yixuan Li"],"abstract":"Out-of-distribution (OOD) detection is important for machine learning models deployed in the wild. Recent methods use auxiliary outlier data to regularize the model for improved OOD detection. However, these approaches make a strong distributional assumption that the auxiliary outlier data is completely separable from the in-distribution (ID) data. In this paper, we propose a novel framework that leverages wild mixture data, which naturally consists of both ID and OOD samples. Such wild data is abundant and arises freely upon deploying a machine learning classifier in their natural habitats. Our key idea is to formulate a constrained optimization problem and to show how to tractably solve it. Our learning objective maximizes the OOD detection rate, subject to constraints on the classification error of ID data and on the OOD error rate of ID examples. We extensively evaluate our approach on common OOD detection tasks and demonstrate superior performance.","url_abs":"https://arxiv.org/abs/2202.03299v2","url_pdf":"https://arxiv.org/pdf/2202.03299v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-ood-detectors-in-their-natural","repo_url":"https://github.com/jkatzsam/woods_ood","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"ood-detection","task_name":"Out of Distribution (OOD) Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.03299","atlas_url":"https://app.syntology.ai/?focus=2202.03299","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.03299"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/jkatzsam/woods_ood","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":7,"ran_honours":1,"unverified":6},"by_repo_kind":{"official":{"samples":14,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"17bbd961aaf86569","entry":"ODIN","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/score_calculation.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/score_calculation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"17bbd961aaf86569"}},{"code_sha256_prefix":"581a8583346c6d6e","entry":"compute_auroc","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"CIFAR/plot_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/CIFAR/plot_results.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"581a8583346c6d6e"}},{"code_sha256_prefix":"7242da839b9822be","entry":"fpr_and_fdr_at_recall","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/display_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/display_results.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7242da839b9822be"}},{"code_sha256_prefix":"24f5b40495bb542e","entry":"get_Mahalanobis_score","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/score_calculation.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/score_calculation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"24f5b40495bb542e"}},{"code_sha256_prefix":"b9b6c36f7a3e7e67","entry":"get_measures","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/display_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/display_results.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b9b6c36f7a3e7e67"}},{"code_sha256_prefix":"3ad01de2f86dccba","entry":"get_ood_scores_odin","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/score_calculation.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/score_calculation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ad01de2f86dccba"}},{"code_sha256_prefix":"d4acb3120a027622","entry":"stable_cumsum","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/display_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/display_results.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d4acb3120a027622"}},{"code_sha256_prefix":"2b5f3b49bd6f6768","entry":"test_fnr_using_valid_threshold","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"CIFAR/plot_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/CIFAR/plot_results.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2b5f3b49bd6f6768"}},{"code_sha256_prefix":"509aa629e46cf950","entry":"calib_err","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/calibration_tools.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/calibration_tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"509aa629e46cf950"}},{"code_sha256_prefix":"2200861da0136416","entry":"load_CIFAR","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"CIFAR/make_datasets.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/CIFAR/make_datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2200861da0136416"}},{"code_sha256_prefix":"b2dd765101838953","entry":"load_results","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"CIFAR/plot_results.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/CIFAR/plot_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b2dd765101838953"}},{"code_sha256_prefix":"0a958e667c8c24c8","entry":"soft_f1","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/calibration_tools.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/calibration_tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a958e667c8c24c8"}},{"code_sha256_prefix":"3c09c6a69241b742","entry":"tune_temp","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/calibration_tools.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/calibration_tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3c09c6a69241b742"}},{"code_sha256_prefix":"4ebc0e09cb353eac","entry":"unpickle","repo":"jkatzsam/woods_ood","repo_kind":"official","path":"utils/imagenet_rc_loader.py","file_url":"https://github.com/jkatzsam/woods_ood/blob/HEAD/utils/imagenet_rc_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ebc0e09cb353eac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}