{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distribution-calibration-for-out-of-domain","title":"Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation","arxiv_id":"2209.06612","date":"2022-09-14","proceeding":"COLING 2022 10","authors":["Yanan Wu","Zhiyuan Zeng","Keqing He","Yutao Mou","Pei Wang","Weiran Xu"],"abstract":"Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. In this paper, we analyze overconfident OOD comes from distribution uncertainty due to the mismatch between the training and test distributions, which makes the model can't confidently make predictions thus probably causing abnormal softmax scores. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. Our method is flexible and easily pluggable into existing softmax-based baselines and gains 33.33\\% OOD F1 improvements with increasing only 0.41\\% inference time compared to MSP. Further analyses show the effectiveness of Bayesian learning for OOD detection.","url_abs":"https://arxiv.org/abs/2209.06612v1","url_pdf":"https://arxiv.org/pdf/2209.06612v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distribution-calibration-for-out-of-domain","repo_url":"https://github.com/pris-nlp/coling2022_bayesian-for-ood","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"ood-detection","task_name":"Out of Distribution (OOD) Detection"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}