{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/two4two-evaluating-interpretable-machine","title":"Two4Two: Evaluating Interpretable Machine Learning - A Synthetic Dataset For Controlled Experiments","arxiv_id":"2105.02825","date":"2021-05-06","proceeding":null,"authors":["Martin Schuessler","Philipp Weiß","Leon Sixt"],"abstract":"A growing number of approaches exist to generate explanations for image classification. However, few of these approaches are subjected to human-subject evaluations, partly because it is challenging to design controlled experiments with natural image datasets, as they leave essential factors out of the researcher's control. With our approach, researchers can describe their desired dataset with only a few parameters. Based on these, our library generates synthetic image data of two 3D abstract animals. The resulting data is suitable for algorithmic as well as human-subject evaluations. Our user study results demonstrate that our method can create biases predictive enough for a classifier and subtle enough to be noticeable only to every second participant inspecting the data visually. Our approach significantly lowers the barrier for conducting human subject evaluations, thereby facilitating more rigorous investigations into interpretable machine learning. For our library and datasets see, https://github.com/mschuessler/two4two/","url_abs":"https://arxiv.org/abs/2105.02825v1","url_pdf":"https://arxiv.org/pdf/2105.02825v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"two4two-evaluating-interpretable-machine","repo_url":"https://github.com/mschuessler/two4two","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"interpretable-machine-learning","task_name":"Interpretable Machine Learning"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.02825","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.02825"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mschuessler/two4two","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"41a508bd03b62b49","entry":"numpy_to_python_scalar","repo":"mschuessler/two4two","repo_kind":"official","path":"two4two/utils.py","file_url":"https://github.com/mschuessler/two4two/blob/HEAD/two4two/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41a508bd03b62b49"}},{"code_sha256_prefix":"c0bc1ef4cfed9a13","entry":"supports_iteration","repo":"mschuessler/two4two","repo_kind":"official","path":"two4two/utils.py","file_url":"https://github.com/mschuessler/two4two/blob/HEAD/two4two/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0bc1ef4cfed9a13"}},{"code_sha256_prefix":"483fdad9d4bd81b7","entry":"truncated_normal","repo":"mschuessler/two4two","repo_kind":"official","path":"two4two/utils.py","file_url":"https://github.com/mschuessler/two4two/blob/HEAD/two4two/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"483fdad9d4bd81b7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}