{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/promises-and-pitfalls-of-black-box-concept","title":"Promises and Pitfalls of Black-Box Concept Learning Models","arxiv_id":"2106.13314","date":"2021-06-24","proceeding":null,"authors":["Anita Mahinpei","Justin Clark","Isaac Lage","Finale Doshi-Velez","Weiwei Pan"],"abstract":"Machine learning models that incorporate concept learning as an intermediate step in their decision making process can match the performance of black-box predictive models while retaining the ability to explain outcomes in human understandable terms. However, we demonstrate that the concept representations learned by these models encode information beyond the pre-defined concepts, and that natural mitigation strategies do not fully work, rendering the interpretation of the downstream prediction misleading. We describe the mechanism underlying the information leakage and suggest recourse for mitigating its effects.","url_abs":"https://arxiv.org/abs/2106.13314v1","url_pdf":"https://arxiv.org/pdf/2106.13314v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"promises-and-pitfalls-of-black-box-concept","repo_url":"https://github.com/mateoespinosa/concept-quality","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"promises-and-pitfalls-of-black-box-concept","repo_url":"https://github.com/tobias-opsahl/hybrid-concept-based-models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.13314","atlas_url":"https://app.syntology.ai/?focus=2106.13314","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}