{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/clarify-improving-model-robustness-with","title":"Clarify: Improving Model Robustness With Natural Language Corrections","arxiv_id":"2402.03715","date":"2024-02-06","proceeding":null,"authors":["Yoonho Lee","Michelle S. Lam","Helena Vasconcelos","Michael S. Bernstein","Chelsea Finn"],"abstract":"The standard way to teach models is by feeding them lots of data. However, this approach often teaches models incorrect ideas because they pick up on misleading signals in the data. To prevent such misconceptions, we must necessarily provide additional information beyond the training data. Prior methods incorporate additional instance-level supervision, such as labels for misleading features or additional labels for debiased data. However, such strategies require a large amount of labeler effort. We hypothesize that people are good at providing textual feedback at the concept level, a capability that existing teaching frameworks do not leverage. We propose Clarify, a novel interface and method for interactively correcting model misconceptions. Through Clarify, users need only provide a short text description of a model's consistent failure patterns. Then, in an entirely automated way, we use such descriptions to improve the training process. Clarify is the first end-to-end system for user model correction. Our user studies show that non-expert users can successfully describe model misconceptions via Clarify, leading to increased worst-case performance in two datasets. We additionally conduct a case study on a large-scale image dataset, ImageNet, using Clarify to find and rectify 31 novel hard subpopulations.","url_abs":"https://arxiv.org/abs/2402.03715v3","url_pdf":"https://arxiv.org/pdf/2402.03715v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"clarify-improving-model-robustness-with","repo_url":"https://github.com/yoonholee/clarify","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"misconceptions","task_name":"Misconceptions"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2402.03715","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03715"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yoonholee/clarify","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"08e22c6216b49d33","entry":"file_cache","repo":"yoonholee/clarify","repo_kind":"official","path":"datasets_all.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/datasets_all.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"08e22c6216b49d33"}},{"code_sha256_prefix":"7a03af682eeae36d","entry":"fill_spurious","repo":"yoonholee/clarify","repo_kind":"official","path":"datasets_all.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/datasets_all.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a03af682eeae36d"}},{"code_sha256_prefix":"acb6e9ac30ac7dbe","entry":"get_error_score","repo":"yoonholee/clarify","repo_kind":"official","path":"server.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/server.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acb6e9ac30ac7dbe"}},{"code_sha256_prefix":"79adb9a597ff434a","entry":"get_group_array","repo":"yoonholee/clarify","repo_kind":"official","path":"datasets_all.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/datasets_all.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79adb9a597ff434a"}},{"code_sha256_prefix":"f67613e242a2602d","entry":"read_h5","repo":"yoonholee/clarify","repo_kind":"official","path":"backbones.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/backbones.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f67613e242a2602d"}},{"code_sha256_prefix":"aacae25f235a12e8","entry":"latent_to_index","repo":"yoonholee/clarify","repo_kind":"official","path":"dsprites.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/dsprites.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aacae25f235a12e8"}},{"code_sha256_prefix":"4f672ac0c8e2d4a0","entry":"sample_latent","repo":"yoonholee/clarify","repo_kind":"official","path":"dsprites.py","file_url":"https://github.com/yoonholee/clarify/blob/HEAD/dsprites.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f672ac0c8e2d4a0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}