{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sum-of-parts-models-faithful-attributions-for","title":"Sum-of-Parts: Faithful Attributions for Groups of Features","arxiv_id":"2310.16316","date":"2023-10-25","proceeding":null,"authors":["Weiqiu You","Helen Qu","Marco Gatti","Bhuvnesh Jain","Eric Wong"],"abstract":"Feature attributions explain machine learning predictions by assigning importance scores to input features. While faithful attributions accurately reflect feature contributions to the model's prediction, unfaithful ones can lead to misleading interpretations, making them unreliable in high-stake domains. The challenge of unfaithfulness of post-hoc attributions led to the development of self-explaining models. However, self-explaining models often trade-off performance for interpretability. In this work, we develop Sum-of-Parts (SOP), a new framework that transforms any differentiable model into a self-explaining model whose predictions can be attributed to groups of features. The SOP framework leverages pretrained deep learning models with custom attention modules to learn useful feature groups end-to-end without direct supervision. With these capabilities, SOP achieves highest performance while also scoring high with respect to faithfulness metrics on both ImageNet and CosmoGrid. We validate the usefulness of the groups learned by SOP through their high purity, strong human distinction ability, and practical utility in scientific discovery. In a case study, we show how SOP assists cosmologists in uncovering new insights about galaxy formation.","url_abs":"https://arxiv.org/abs/2310.16316v2","url_pdf":"https://arxiv.org/pdf/2310.16316v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sum-of-parts-models-faithful-attributions-for","repo_url":"https://github.com/brachiolab/sop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sum-of-parts-models-faithful-attributions-for","repo_url":"https://github.com/debugml/sop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"scientific-discovery","task_name":"scientific discovery"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.16316","atlas_url":"https://app.syntology.ai/?focus=2310.16316","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.16316"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/debugml/sop","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/brachiolab/sop","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"dc15db7b76e4f0ee","entry":"WrappedModel","repo":"brachiolab/sop","repo_kind":"official","path":"src/sop/tasks/images/cosmogrid/models.py","file_url":"https://github.com/brachiolab/sop/blob/HEAD/src/sop/tasks/images/cosmogrid/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc15db7b76e4f0ee"}},{"code_sha256_prefix":"2843042d2c0e6f50","entry":"annotations_from_jsonl","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/eraser_utils.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/eraser_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2843042d2c0e6f50"}},{"code_sha256_prefix":"e1878c4a3bf022ae","entry":"load_datasets","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/eraser_utils.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/eraser_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e1878c4a3bf022ae"}},{"code_sha256_prefix":"49077456b34beefd","entry":"load_jsonl","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/eraser_utils.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/eraser_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49077456b34beefd"}},{"code_sha256_prefix":"1040c2d9a4545271","entry":"sum_weights_for_unique_masks","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/cosmogrid_acc_purity.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/cosmogrid_acc_purity.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1040c2d9a4545271"}},{"code_sha256_prefix":"dc13a5e3d052bf6a","entry":"get_all_fidelity","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/fidelity.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/fidelity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc13a5e3d052bf6a"}},{"code_sha256_prefix":"6e8c4a5716a9e055","entry":"get_all_fidelity_cosmo","repo":"debugml/sop","repo_kind":"official","path":"src/sop/metrics/cosmogrid_fidelity.py","file_url":"https://github.com/debugml/sop/blob/HEAD/src/sop/metrics/cosmogrid_fidelity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e8c4a5716a9e055"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}