{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/using-explanations-to-guide-models","title":"Studying How to Efficiently and Effectively Guide Models with Explanations","arxiv_id":"2303.11932","date":"2023-03-21","proceeding":"ICCV 2023 1","authors":["Sukrut Rao","Moritz Böhle","Amin Parchami-Araghi","Bernt Schiele"],"abstract":"Despite being highly performant, deep neural networks might base their decisions on features that spuriously correlate with the provided labels, thus hurting generalization. To mitigate this, 'model guidance' has recently gained popularity, i.e. the idea of regularizing the models' explanations to ensure that they are \"right for the right reasons\". While various techniques to achieve such model guidance have been proposed, experimental validation of these approaches has thus far been limited to relatively simple and / or synthetic datasets. To better understand the effectiveness of the various design choices that have been explored in the context of model guidance, in this work we conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets. As annotation costs for model guidance can limit its applicability, we also place a particular focus on efficiency. Specifically, we guide the models via bounding box annotations, which are much cheaper to obtain than the commonly used segmentation masks, and evaluate the robustness of model guidance under limited (e.g. with only 1% of annotated images) or overly coarse annotations. Further, we propose using the EPG score as an additional evaluation metric and loss function ('Energy loss'). We show that optimizing for the Energy loss leads to models that exhibit a distinct focus on object-specific features, despite only using bounding box annotations that also include background regions. Lastly, we show that such model guidance can improve generalization under distribution shifts. Code available at: https://github.com/sukrutrao/Model-Guidance.","url_abs":"https://arxiv.org/abs/2303.11932v2","url_pdf":"https://arxiv.org/pdf/2303.11932v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"using-explanations-to-guide-models","repo_url":"https://github.com/sukrutrao/model-guidance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.11932","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.11932"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sukrutrao/model-guidance","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/sukrutrao/Model-Guidance","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"fixup_resnet.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/fixup_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"fixup_resnet.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/fixup_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"b43d06a762f0ab3a","entry":"filter_bbs","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"utils.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b43d06a762f0ab3a"}},{"code_sha256_prefix":"2b66d0ad17b50dad","entry":"get_attributor","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"attribution_methods.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/attribution_methods.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b66d0ad17b50dad"}},{"code_sha256_prefix":"455c0b7e3b5331a4","entry":"get_localization_loss","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"losses.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"455c0b7e3b5331a4"}},{"code_sha256_prefix":"ead3249bfdca88d2","entry":"get_random_optimization_targets","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"utils.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ead3249bfdca88d2"}},{"code_sha256_prefix":"7530661969b12565","entry":"remove_module","repo":"sukrutrao/Model-Guidance","repo_kind":"official","path":"utils.py","file_url":"https://github.com/sukrutrao/Model-Guidance/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7530661969b12565"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}