{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mapping-the-mind-of-an-instruction-based","title":"Mapping the Mind of an Instruction-based Image Editing using SMILE","arxiv_id":"2412.16277","date":"2024-12-20","proceeding":null,"authors":["Zeinab Dehghani","Koorosh Aslansefat","Adil Khan","Adín Ramírez Rivera","Franky George","Muhammad Khalid"],"abstract":"Despite recent advancements in Instruct-based Image Editing models for generating high-quality images, they are known as black boxes and a significant barrier to transparency and user trust. To solve this issue, we introduce SMILE (Statistical Model-agnostic Interpretability with Local Explanations), a novel model-agnostic for localized interpretability that provides a visual heatmap to clarify the textual elements' influence on image-generating models. We applied our method to various Instruction-based Image Editing models like Pix2Pix, Image2Image-turbo and Diffusers-Inpaint and showed how our model can improve interpretability and reliability. Also, we use stability, accuracy, fidelity, and consistency metrics to evaluate our method. These findings indicate the exciting potential of model-agnostic interpretability for reliability and trustworthiness in critical applications such as healthcare and autonomous driving while encouraging additional investigation into the significance of interpretability in enhancing dependable image editing models.","url_abs":"https://arxiv.org/abs/2412.16277v1","url_pdf":"https://arxiv.org/pdf/2412.16277v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mapping-the-mind-of-an-instruction-based","repo_url":"https://github.com/dependable-intelligent-systems-lab/xwhy","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"mapping-the-mind-of-an-instruction-based","repo_url":"https://github.com/rishidarkdevil/daam-i2i","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"heatmap","method_name":"Heatmap"},{"method_slug":"patchgan","method_name":"PatchGAN"},{"method_slug":"pix2pix","method_name":"Pix2Pix"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}