{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/amazon-sagemaker-clarify-machine-learning","title":"Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud","arxiv_id":"2109.03285","date":"2021-09-07","proceeding":null,"authors":["Michaela Hardt","Xiaoguang Chen","Xiaoyi Cheng","Michele Donini","Jason Gelman","Satish Gollaprolu","John He","Pedro Larroy","Xinyu Liu","Nick McCarthy","Ashish Rathi","Scott Rees","Ankit Siva","ErhYuan Tsai","Keerthan Vasist","Pinar Yilmaz","Muhammad Bilal Zafar","Sanjiv Das","Kevin Haas","Tyler Hill","Krishnaram Kenthapadi"],"abstract":"Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining predictions. It is deeply integrated into Amazon SageMaker, a fully managed service that enables data scientists and developers to build, train, and deploy ML models at any scale. Clarify supports bias detection and feature importance computation across the ML lifecycle, during data preparation, model evaluation, and post-deployment monitoring. We outline the desiderata derived from customer input, the modular architecture, and the methodology for bias and explanation computations. Further, we describe the technical challenges encountered and the tradeoffs we had to make. For illustration, we discuss two customer use cases. We present our deployment results including qualitative customer feedback and a quantitative evaluation. Finally, we summarize lessons learned, and discuss best practices for the successful adoption of fairness and explanation tools in practice.","url_abs":"https://arxiv.org/abs/2109.03285v1","url_pdf":"https://arxiv.org/pdf/2109.03285v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"amazon-sagemaker-clarify-machine-learning","repo_url":"https://github.com/aws/amazon-sagemaker-clarify","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"bias-detection","task_name":"Bias Detection"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"feature-importance","task_name":"Feature Importance"}],"methods":[{"method_slug":null,"method_name":null}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.03285","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.03285"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aws/amazon-sagemaker-clarify","reach":null}],"summary":{"ran_violates":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3d382eb18ecc9048","entry":"proportion","repo":"aws/amazon-sagemaker-clarify","repo_kind":"official","path":"src/smclarify/bias/metrics/basic_stats.py","file_url":"https://github.com/aws/amazon-sagemaker-clarify/blob/HEAD/src/smclarify/bias/metrics/basic_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d382eb18ecc9048"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}