{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decoupling-pixel-flipping-and-occlusion","title":"Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks","arxiv_id":"2401.06654","date":"2024-01-12","proceeding":null,"authors":["Stefan Blücher","Johanna Vielhaben","Nils Strodthoff"],"abstract":"Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.","url_abs":"https://arxiv.org/abs/2401.06654v1","url_pdf":"https://arxiv.org/pdf/2401.06654v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decoupling-pixel-flipping-and-occlusion","repo_url":"https://github.com/bluecher31/pixel-flipping","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"LGPL-2.1"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pixel-prediction","method_name":"Inpainting"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.06654","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.06654"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bluecher31/pixel-flipping","reach":{"status":"ok","spdx":"LGPL-2.1"}}],"summary":{"ran":15,"unverified":3},"by_repo_kind":{"official":{"samples":18,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":18,"samples":[{"code_sha256_prefix":"91b9e7125ceb0cf4","entry":"calc_n_interactions","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_methods.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_methods.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"91b9e7125ceb0cf4"}},{"code_sha256_prefix":"d09d3325aee96b41","entry":"calculate_auc","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/plotting.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"d09d3325aee96b41"}},{"code_sha256_prefix":"ff9025c47e8c0500","entry":"calculate_fig_size_in_inches","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/config_matplotlib.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/config_matplotlib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"ff9025c47e8c0500"}},{"code_sha256_prefix":"9985fb8a9aa23137","entry":"convert_heatmap_to_superpixels","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/experiments/pixel_flipping.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/experiments/pixel_flipping.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"9985fb8a9aa23137"}},{"code_sha256_prefix":"d3a250b3a06ee085","entry":"convert_images_to_array","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/interface.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/interface.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"d3a250b3a06ee085"}},{"code_sha256_prefix":"7b87d4d8f34d0f7e","entry":"generate_feature_interaction_algebra","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_methods.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_methods.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"7b87d4d8f34d0f7e"}},{"code_sha256_prefix":"fdf5b2bff88d4d30","entry":"generate_footprints","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_methods.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_methods.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"fdf5b2bff88d4d30"}},{"code_sha256_prefix":"97206f44efbf5255","entry":"get_and_check_shapes","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_base.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"97206f44efbf5255"}},{"code_sha256_prefix":"5d6f5c141b36c5f5","entry":"laplace_smoothing","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_base.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"5d6f5c141b36c5f5"}},{"code_sha256_prefix":"1f27b7255efddbe4","entry":"line_with_shaded_errorband","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/config_matplotlib.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/config_matplotlib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"1f27b7255efddbe4"}},{"code_sha256_prefix":"9d7dfa1a06f1cd38","entry":"root_pixel_flipping","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/experiments/pixel_flipping.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/experiments/pixel_flipping.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"9d7dfa1a06f1cd38"}},{"code_sha256_prefix":"c6a5d61af67dff46","entry":"select_label","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/experiments/pixel_flipping.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/experiments/pixel_flipping.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"c6a5d61af67dff46"}},{"code_sha256_prefix":"eb30d71d3bb813b8","entry":"shap_kernel_weight","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_kernelshap.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_kernelshap.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"eb30d71d3bb813b8"}},{"code_sha256_prefix":"0731bd35bbdec170","entry":"shapley_normalization","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/approximate_shapley_values.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/approximate_shapley_values.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"0731bd35bbdec170"}},{"code_sha256_prefix":"2a6d7f87e2880716","entry":"summarize_rgb_channels","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"src/plotting.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/src/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"2a6d7f87e2880716"}},{"code_sha256_prefix":"c8d48ee3f9addbbf","entry":"organize_key","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_base.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"c8d48ee3f9addbbf"}},{"code_sha256_prefix":"e39eec6fdb523290","entry":"power_set_coalitions","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_kernelshap.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_kernelshap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"e39eec6fdb523290"}},{"code_sha256_prefix":"4f993179ee2018b8","entry":"random_coalitions","repo":"bluecher31/pixel-flipping","repo_kind":"official","path":"conditional_explainer/helper_kernelshap.py","file_url":"https://github.com/bluecher31/pixel-flipping/blob/HEAD/conditional_explainer/helper_kernelshap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-2.1","inline_ok":false,"mcp_get_code":{"code_sha256":"4f993179ee2018b8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}