{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/open-set-recognition-a-good-closed-set-1","title":"Open-Set Recognition: a Good Closed-Set Classifier is All You Need?","arxiv_id":"2110.06207","date":"2021-10-12","proceeding":"ICLR 2022 4","authors":["Sagar Vaze","Kai Han","Andrea Vedaldi","Andrew Zisserman"],"abstract":"The ability to identify whether or not a test sample belongs to one of the semantic classes in a classifier's training set is critical to practical deployment of the model. This task is termed open-set recognition (OSR) and has received significant attention in recent years. In this paper, we first demonstrate that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We find that this relationship holds across loss objectives and architectures, and further demonstrate the trend both on the standard OSR benchmarks as well as on a large-scale ImageNet evaluation. Second, we use this correlation to boost the performance of a maximum logit score OSR 'baseline' by improving its closed-set accuracy, and with this strong baseline achieve state-of-the-art on a number of OSR benchmarks. Similarly, we boost the performance of the existing state-of-the-art method by improving its closed-set accuracy, but the resulting discrepancy with the strong baseline is marginal. Our third contribution is to present the 'Semantic Shift Benchmark' (SSB), which better respects the task of detecting semantic novelty, in contrast to other forms of distribution shift also considered in related sub-fields, such as out-of-distribution detection. On this new evaluation, we again demonstrate that there is negligible difference between the strong baseline and the existing state-of-the-art. Project Page: https://www.robots.ox.ac.uk/~vgg/research/osr/","url_abs":"https://arxiv.org/abs/2110.06207v2","url_pdf":"https://arxiv.org/pdf/2110.06207v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"open-set-recognition-a-good-closed-set-1","repo_url":"https://github.com/sgvaze/osr_closed_set_all_you_need","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"open-set-recognition-a-good-closed-set-1","repo_url":"https://github.com/t9liverpool/OpenMix","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"open-set-learning","task_name":"Open Set Learning"},{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/out-of-distribution-detection-on-cifar-100-vs","task":"Out-of-Distribution Detection","dataset":"CIFAR-100 vs CIFAR-10","model":"Maximum Logit Score (MLS)","rank_in_archive_order":10,"of":14,"metrics":{"AUROC":"83.2"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.06207","atlas_url":"https://app.syntology.ai/?focus=2110.06207","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.06207"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sgvaze/osr_closed_set_all_you_need","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/t9liverpool/OpenMix","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran":9,"ran_fixture":1,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":17,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c62a45794431e9b7","entry":"get_mean_lr","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/osr.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/osr.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c62a45794431e9b7"}},{"code_sha256_prefix":"3cce4936256bdc33","entry":"Discriminator32","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/arpl_models/gan.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/arpl_models/gan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cce4936256bdc33"}},{"code_sha256_prefix":"1a6a50a611f464f9","entry":"Generator","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/arpl_models/gan.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/arpl_models/gan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a6a50a611f464f9"}},{"code_sha256_prefix":"8c24b39fc0eecb58","entry":"Generator32","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/arpl_models/gan.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/arpl_models/gan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c24b39fc0eecb58"}},{"code_sha256_prefix":"d3319e3d34ca90ca","entry":"drop_connect","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/miscel_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/miscel_utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3319e3d34ca90ca"}},{"code_sha256_prefix":"ed5db303ab193775","entry":"get_optimizer","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/osr.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/osr.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed5db303ab193775"}},{"code_sha256_prefix":"a811aa6eac543d8f","entry":"load_networks","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"methods/ARPL/arpl_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/methods/ARPL/arpl_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a811aa6eac543d8f"}},{"code_sha256_prefix":"f15a49337e69e937","entry":"round_filters","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/miscel_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/miscel_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f15a49337e69e937"}},{"code_sha256_prefix":"dbc0ca08d119a5a0","entry":"round_repeats","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/miscel_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/miscel_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbc0ca08d119a5a0"}},{"code_sha256_prefix":"bc2c748ee9fffddf","entry":"strip_state_dict","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc2c748ee9fffddf"}},{"code_sha256_prefix":"443a6ab0aca3d0e3","entry":"transform_moco_state_dict","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/model_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"443a6ab0aca3d0e3"}},{"code_sha256_prefix":"d040dabdf943f132","entry":"transform_moco_state_dict_arpl_cs","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/model_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d040dabdf943f132"}},{"code_sha256_prefix":"1219ffd1eb121b9e","entry":"transform_moco_state_dict_places","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"models/model_utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/models/model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1219ffd1eb121b9e"}},{"code_sha256_prefix":"913d82af53065418","entry":"accuracy","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"913d82af53065418"}},{"code_sha256_prefix":"142c0ce9982cbf82","entry":"get_file","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"utils/logfile_parser.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/utils/logfile_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"142c0ce9982cbf82"}},{"code_sha256_prefix":"8d59d5daedcdf3f8","entry":"get_scheduler","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"utils/schedulers.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/utils/schedulers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d59d5daedcdf3f8"}},{"code_sha256_prefix":"7767f8e34fb9f60c","entry":"init_experiment","repo":"sgvaze/osr_closed_set_all_you_need","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/sgvaze/osr_closed_set_all_you_need/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7767f8e34fb9f60c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}