{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evaluating-weakly-supervised-object","title":"Evaluating Weakly Supervised Object Localization Methods Right","arxiv_id":"2001.07437","date":"2020-01-21","proceeding":"CVPR 2020 6","authors":["Junsuk Choe","Seong Joon Oh","Seungho Lee","Sanghyuk Chun","Zeynep Akata","Hyunjung Shim"],"abstract":"Weakly-supervised object localization (WSOL) has gained popularity over the last years for its promise to train localization models with only image-level labels. Since the seminal WSOL work of class activation mapping (CAM), the field has focused on how to expand the attention regions to cover objects more broadly and localize them better. However, these strategies rely on full localization supervision to validate hyperparameters and for model selection, which is in principle prohibited under the WSOL setup. In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set. We observe that, under our protocol, the five most recent WSOL methods have not made a major improvement over the CAM baseline. Moreover, we report that existing WSOL methods have not reached the few-shot learning baseline, where the full-supervision at validation time is used for model training instead. Based on our findings, we discuss some future directions for WSOL.","url_abs":"https://arxiv.org/abs/2001.07437v2","url_pdf":"https://arxiv.org/pdf/2001.07437v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evaluating-weakly-supervised-object","repo_url":"https://github.com/clovaai/wsolevaluation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"evaluating-weakly-supervised-object","repo_url":"https://github.com/GenDisc/IVR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-localization","task_name":"Object Localization"},{"task_slug":"weakly-supervised-object-localization","task_name":"Weakly-Supervised Object Localization"}],"methods":[{"method_slug":"cam","method_name":"CAM"},{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2001.07437","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2001.07437"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GenDisc/IVR","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/clovaai/wsolevaluation","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":1,"ran_violates":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c2fedf58aea8b2ba","entry":"get_architecture_type","repo":"GenDisc/IVR","repo_kind":"listed","path":"config.py","file_url":"https://github.com/GenDisc/IVR/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2fedf58aea8b2ba"}},{"code_sha256_prefix":"f017532fc389cbfe","entry":"str2bool","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f017532fc389cbfe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}