{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-what-if-tool-interactive-probing-of","title":"The What-If Tool: Interactive Probing of Machine Learning Models","arxiv_id":"1907.04135","date":"2019-07-09","proceeding":null,"authors":["James Wexler","Mahima Pushkarna","Tolga Bolukbasi","Martin Wattenberg","Fernanda Viegas","Jimbo Wilson"],"abstract":"A key challenge in developing and deploying Machine Learning (ML) systems is understanding their performance across a wide range of inputs. To address this challenge, we created the What-If Tool, an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding. The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics. We describe the design of the tool, and report on real-life usage at different organizations.","url_abs":"https://arxiv.org/abs/1907.04135v2","url_pdf":"https://arxiv.org/pdf/1907.04135v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-what-if-tool-interactive-probing-of","repo_url":"https://github.com/pair-code/what-if-tool","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1907.04135","atlas_url":"https://app.syntology.ai/?focus=1907.04135","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1907.04135"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pair-code/what-if-tool","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c09d84260bbe1825","entry":"ensure_not_binary","repo":"pair-code/what-if-tool","repo_kind":"official","path":"utils/inference_utils.py","file_url":"https://github.com/pair-code/what-if-tool/blob/HEAD/utils/inference_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c09d84260bbe1825"}},{"code_sha256_prefix":"8312c60a13f07762","entry":"filepath_to_filepath_list","repo":"pair-code/what-if-tool","repo_kind":"official","path":"utils/platform_utils.py","file_url":"https://github.com/pair-code/what-if-tool/blob/HEAD/utils/platform_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8312c60a13f07762"}},{"code_sha256_prefix":"432d95cbff7f6c50","entry":"parse_original_feature_from_example","repo":"pair-code/what-if-tool","repo_kind":"official","path":"utils/inference_utils.py","file_url":"https://github.com/pair-code/what-if-tool/blob/HEAD/utils/inference_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"432d95cbff7f6c50"}},{"code_sha256_prefix":"02df712a578b5861","entry":"path_is_parent","repo":"pair-code/what-if-tool","repo_kind":"official","path":"utils/platform_utils.py","file_url":"https://github.com/pair-code/what-if-tool/blob/HEAD/utils/platform_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"02df712a578b5861"}},{"code_sha256_prefix":"e07682087b90dcf9","entry":"proto_value_for_feature","repo":"pair-code/what-if-tool","repo_kind":"official","path":"utils/inference_utils.py","file_url":"https://github.com/pair-code/what-if-tool/blob/HEAD/utils/inference_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e07682087b90dcf9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}