{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/focus-familiar-objects-in-common-and-uncommon","title":"FOCUS: Familiar Objects in Common and Uncommon Settings","arxiv_id":"2110.03804","date":"2021-10-07","proceeding":null,"authors":["Priyatham Kattakinda","Soheil Feizi"],"abstract":"Standard training datasets for deep learning often contain objects in common settings (e.g., \"a horse on grass\" or \"a ship in water\") since they are usually collected by randomly scraping the web. Uncommon and rare settings (e.g., \"a plane on water\", \"a car in snowy weather\") are thus severely under-represented in the training data. This can lead to an undesirable bias in model predictions towards common settings and create a false sense of accuracy. In this paper, we introduce FOCUS (Familiar Objects in Common and Uncommon Settings), a dataset for stress-testing the generalization power of deep image classifiers. By leveraging the power of modern search engines, we deliberately gather data containing objects in common and uncommon settings in a wide range of locations, weather conditions, and time of day. We present a detailed analysis of the performance of various popular image classifiers on our dataset and demonstrate a clear drop in performance when classifying images in uncommon settings. By analyzing deep features of these models, we show that such errors can be due to the use of spurious features in model predictions. We believe that our dataset will aid researchers in understanding the inability of deep models to generalize well to uncommon settings and drive future work on improving their distributional robustness.","url_abs":"https://arxiv.org/abs/2110.03804v2","url_pdf":"https://arxiv.org/pdf/2110.03804v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"focus-familiar-objects-in-common-and-uncommon","repo_url":"https://github.com/priyathamkat/focus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.03804","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.03804"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/openai/CLIP","reach":{"status":"unanswered"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/priyathamkat/focus","reach":null}],"summary":{"ran_fixture":1,"ran_violates":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"65266aa5d6ea0dea","entry":"check_label","repo":"priyathamkat/focus","repo_kind":"official","path":"src/experiments/evaluate_model.py","file_url":"https://github.com/priyathamkat/focus/blob/HEAD/src/experiments/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"65266aa5d6ea0dea"}},{"code_sha256_prefix":"49b1361be4dc42fd","entry":"is_time_uncommon","repo":"priyathamkat/focus","repo_kind":"official","path":"src/experiments/evaluate_model.py","file_url":"https://github.com/priyathamkat/focus/blob/HEAD/src/experiments/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"49b1361be4dc42fd"}},{"code_sha256_prefix":"4ec8689e6e353fb4","entry":"split_dataset","repo":"priyathamkat/focus","repo_kind":"official","path":"src/experiments/dataset.py","file_url":"https://github.com/priyathamkat/focus/blob/HEAD/src/experiments/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4ec8689e6e353fb4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}