{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aha-human-assisted-out-of-distribution","title":"AHA: Human-Assisted Out-of-Distribution Generalization and Detection","arxiv_id":"2410.08000","date":"2024-10-10","proceeding":null,"authors":["Haoyue Bai","Jifan Zhang","Robert Nowak"],"abstract":"Modern machine learning models deployed often encounter distribution shifts in real-world applications, manifesting as covariate or semantic out-of-distribution (OOD) shifts. These shifts give rise to challenges in OOD generalization and OOD detection. This paper introduces a novel, integrated approach AHA (Adaptive Human-Assisted OOD learning) to simultaneously address both OOD generalization and detection through a human-assisted framework by labeling data in the wild. Our approach strategically labels examples within a novel maximum disambiguation region, where the number of semantic and covariate OOD data roughly equalizes. By labeling within this region, we can maximally disambiguate the two types of OOD data, thereby maximizing the utility of the fixed labeling budget. Our algorithm first utilizes a noisy binary search algorithm that identifies the maximal disambiguation region with high probability. The algorithm then continues with annotating inside the identified labeling region, reaping the full benefit of human feedback. Extensive experiments validate the efficacy of our framework. We observed that with only a few hundred human annotations, our method significantly outperforms existing state-of-the-art methods that do not involve human assistance, in both OOD generalization and OOD detection. Code is publicly available at \\url{https://github.com/HaoyueBaiZJU/aha}.","url_abs":"https://arxiv.org/abs/2410.08000v1","url_pdf":"https://arxiv.org/pdf/2410.08000v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2410.08000","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.08000"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/HaoyueBaiZJU/aha","reach":{"status":"ok"}}],"summary":{"ran":2,"ran_honours":1},"by_repo_kind":{"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"1134ab62338793e2","entry":"fpr_and_fdr_at_recall","repo":"HaoyueBaiZJU/aha","repo_kind":"found_in_text","path":"utils/metric_utils.py","file_url":"https://github.com/HaoyueBaiZJU/aha/blob/HEAD/utils/metric_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1134ab62338793e2"}},{"code_sha256_prefix":"9b1abfca5a67be06","entry":"get_measures","repo":"HaoyueBaiZJU/aha","repo_kind":"found_in_text","path":"utils/metric_utils.py","file_url":"https://github.com/HaoyueBaiZJU/aha/blob/HEAD/utils/metric_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b1abfca5a67be06"}},{"code_sha256_prefix":"d4acb3120a027622","entry":"stable_cumsum","repo":"HaoyueBaiZJU/aha","repo_kind":"found_in_text","path":"utils/metric_utils.py","file_url":"https://github.com/HaoyueBaiZJU/aha/blob/HEAD/utils/metric_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4acb3120a027622"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}