{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/shortcutprobe-probing-prediction-shortcuts","title":"ShortcutProbe: Probing Prediction Shortcuts for Learning Robust Models","arxiv_id":"2505.13910","date":"2025-05-20","proceeding":null,"authors":["Guangtao Zheng","Wenqian Ye","Aidong Zhang"],"abstract":"Deep learning models often achieve high performance by inadvertently learning spurious correlations between targets and non-essential features. For example, an image classifier may identify an object via its background that spuriously correlates with it. This prediction behavior, known as spurious bias, severely degrades model performance on data that lacks the learned spurious correlations. Existing methods on spurious bias mitigation typically require a variety of data groups with spurious correlation annotations called group labels. However, group labels require costly human annotations and often fail to capture subtle spurious biases such as relying on specific pixels for predictions. In this paper, we propose a novel post hoc spurious bias mitigation framework without requiring group labels. Our framework, termed ShortcutProbe, identifies prediction shortcuts that reflect potential non-robustness in predictions in a given model's latent space. The model is then retrained to be invariant to the identified prediction shortcuts for improved robustness. We theoretically analyze the effectiveness of the framework and empirically demonstrate that it is an efficient and practical tool for improving a model's robustness to spurious bias on diverse datasets.","url_abs":"https://arxiv.org/abs/2505.13910v1","url_pdf":"https://arxiv.org/pdf/2505.13910v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"shortcutprobe-probing-prediction-shortcuts","repo_url":"https://github.com/gtzheng/shortcutprobe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"hoc","method_name":"HOC"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.13910","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.13910"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gtzheng/shortcutprobe","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/gtzheng/ShortcutProbe","reach":{"status":"ok"}}],"summary":{"ran":14,"ran_draft_wrong":6,"ran_honours":1,"unverified":6},"by_repo_kind":{"official":{"samples":27,"ran":21,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":27,"samples":[{"code_sha256_prefix":"6e663667ba7ed3f2","entry":"ClassBalancedSampler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e663667ba7ed3f2"}},{"code_sha256_prefix":"a87713c5b6ae5b15","entry":"EmbedDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a87713c5b6ae5b15"}},{"code_sha256_prefix":"c468389298284a04","entry":"GroupBalancedSampler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c468389298284a04"}},{"code_sha256_prefix":"4711b44eb7381e64","entry":"IdxDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4711b44eb7381e64"}},{"code_sha256_prefix":"fa1c55aa70701d1e","entry":"JointDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa1c55aa70701d1e"}},{"code_sha256_prefix":"a5207b840e3096a2","entry":"JointRandomSampler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5207b840e3096a2"}},{"code_sha256_prefix":"52fb87f5341104be","entry":"MisDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52fb87f5341104be"}},{"code_sha256_prefix":"2ee64f5afd17a5a3","entry":"PredDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2ee64f5afd17a5a3"}},{"code_sha256_prefix":"36087249bd727695","entry":"RandomSampler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36087249bd727695"}},{"code_sha256_prefix":"4a0b5cf2bf60d9e7","entry":"RunningVariance","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a0b5cf2bf60d9e7"}},{"code_sha256_prefix":"f6cdc99333d7ac0c","entry":"ShortcutSampler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6cdc99333d7ac0c"}},{"code_sha256_prefix":"b3e59759edda6308","entry":"SpuriousVectors","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3e59759edda6308"}},{"code_sha256_prefix":"3be6f85f9a2c50c3","entry":"SubsetDataset","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3be6f85f9a2c50c3"}},{"code_sha256_prefix":"abb051ad54c04456","entry":"bert_adamw_optimizer","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"abb051ad54c04456"}},{"code_sha256_prefix":"227d71f4a9526c48","entry":"disentangle_pca","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"227d71f4a9526c48"}},{"code_sha256_prefix":"ffd024417d2e0ee9","entry":"extract_feature_info","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffd024417d2e0ee9"}},{"code_sha256_prefix":"24df1f39ca418afa","entry":"get_split_indexes","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24df1f39ca418afa"}},{"code_sha256_prefix":"ad87b4581c72eb68","entry":"init_optimizer","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad87b4581c72eb68"}},{"code_sha256_prefix":"97a6fbbb8298e7a0","entry":"init_scheduler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97a6fbbb8298e7a0"}},{"code_sha256_prefix":"2a04614f7104524c","entry":"load_embeddings","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a04614f7104524c"}},{"code_sha256_prefix":"6f906e5668960d23","entry":"register_algorithm","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f906e5668960d23"}},{"code_sha256_prefix":"9c581ceeab1e7194","entry":"Algorithm","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9c581ceeab1e7194"}},{"code_sha256_prefix":"aa528f9aa86d7413","entry":"Classifier","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa528f9aa86d7413"}},{"code_sha256_prefix":"6a3dd1d39f936e97","entry":"Scaler","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a3dd1d39f936e97"}},{"code_sha256_prefix":"3db8286f1e2584d6","entry":"get_backbone","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3db8286f1e2584d6"}},{"code_sha256_prefix":"0d22a2219be7c189","entry":"prepare_data","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d22a2219be7c189"}},{"code_sha256_prefix":"d99a2ddd28ee9061","entry":"shortcutprobe","repo":"gtzheng/ShortcutProbe","repo_kind":"official","path":"algorithms/shortcut_probe.py","file_url":"https://github.com/gtzheng/ShortcutProbe/blob/HEAD/algorithms/shortcut_probe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d99a2ddd28ee9061"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}