{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/peeb-part-based-image-classifiers-with-an","title":"PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck","arxiv_id":"2403.05297","date":"2024-03-08","proceeding":null,"authors":["Thang M. Pham","Peijie Chen","Tin Nguyen","Seunghyun Yoon","Trung Bui","Anh Totti Nguyen"],"abstract":"CLIP-based classifiers rely on the prompt containing a {class name} that is known to the text encoder. Therefore, they perform poorly on new classes or the classes whose names rarely appear on the Internet (e.g., scientific names of birds). For fine-grained classification, we propose PEEB - an explainable and editable classifier to (1) express the class name into a set of text descriptors that describe the visual parts of that class; and (2) match the embeddings of the detected parts to their textual descriptors in each class to compute a logit score for classification. In a zero-shot setting where the class names are unknown, PEEB outperforms CLIP by a huge margin (~10x in top-1 accuracy). Compared to part-based classifiers, PEEB is not only the state-of-the-art (SOTA) on the supervised-learning setting (88.80% and 92.20% accuracy on CUB-200 and Dogs-120, respectively) but also the first to enable users to edit the text descriptors to form a new classifier without any re-training. Compared to concept bottleneck models, PEEB is also the SOTA in both zero-shot and supervised-learning settings.","url_abs":"https://arxiv.org/abs/2403.05297v3","url_pdf":"https://arxiv.org/pdf/2403.05297v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"peeb-part-based-image-classifiers-with-an","repo_url":"https://github.com/anguyen8/peeb","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.05297","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.05297"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anguyen8/peeb","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":12,"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":21,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a1dd2113af7b709d","entry":"bound_boxes","repo":"anguyen8/peeb","repo_kind":"official","path":"src/precompute_boxes.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/precompute_boxes.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1dd2113af7b709d"}},{"code_sha256_prefix":"2ff51a5d9e6dcdf3","entry":"check_exist","repo":"anguyen8/peeb","repo_kind":"official","path":"src/precompute_boxes.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/precompute_boxes.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ff51a5d9e6dcdf3"}},{"code_sha256_prefix":"8fa582c5f6fddaa6","entry":"generate_prompt_sachit","repo":"anguyen8/peeb","repo_kind":"official","path":"src/chatgpt.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/chatgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8fa582c5f6fddaa6"}},{"code_sha256_prefix":"0988460641132af1","entry":"get_data_split","repo":"anguyen8/peeb","repo_kind":"official","path":"src/train_owlvit.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/train_owlvit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0988460641132af1"}},{"code_sha256_prefix":"61d607c1eb3ac510","entry":"get_prompt_list","repo":"anguyen8/peeb","repo_kind":"official","path":"src/prompt_engineering.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/prompt_engineering.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61d607c1eb3ac510"}},{"code_sha256_prefix":"924c3ad69ec6a8fa","entry":"img_transform","repo":"anguyen8/peeb","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"924c3ad69ec6a8fa"}},{"code_sha256_prefix":"154c00edfc8d8524","entry":"load_descriptions","repo":"anguyen8/peeb","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"154c00edfc8d8524"}},{"code_sha256_prefix":"d3faa0ed46f2e436","entry":"nested_tensor_from_tensor_list","repo":"anguyen8/peeb","repo_kind":"official","path":"src/owlvit_cls.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/owlvit_cls.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3faa0ed46f2e436"}},{"code_sha256_prefix":"e54dd17857816dda","entry":"nested_tensor_from_tensor_list","repo":"anguyen8/peeb","repo_kind":"official","path":"src/owlvit_inference.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/owlvit_inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e54dd17857816dda"}},{"code_sha256_prefix":"334d1b6d7376281e","entry":"rgb_to_hex","repo":"anguyen8/peeb","repo_kind":"official","path":"src/misc/peeb_plots.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/misc/peeb_plots.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"334d1b6d7376281e"}},{"code_sha256_prefix":"695ff74df95375df","entry":"sigmoid_focal_loss","repo":"anguyen8/peeb","repo_kind":"official","path":"src/owlvit_cls.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/owlvit_cls.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"695ff74df95375df"}},{"code_sha256_prefix":"1812ba359126089a","entry":"wordify","repo":"anguyen8/peeb","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1812ba359126089a"}},{"code_sha256_prefix":"dd6b921da641c598","entry":"xy_in_box","repo":"anguyen8/peeb","repo_kind":"official","path":"src/precompute_boxes.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/precompute_boxes.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd6b921da641c598"}},{"code_sha256_prefix":"21070f8951fe8f1f","entry":"add_extra_negatives","repo":"anguyen8/peeb","repo_kind":"official","path":"src/train_owlvit.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/train_owlvit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21070f8951fe8f1f"}},{"code_sha256_prefix":"ca18559102973d9c","entry":"dice_loss","repo":"anguyen8/peeb","repo_kind":"official","path":"src/owlvit_cls.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/owlvit_cls.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca18559102973d9c"}},{"code_sha256_prefix":"5088107d67670278","entry":"fig_to_pil","repo":"anguyen8/peeb","repo_kind":"official","path":"src/misc/peeb_plots.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/misc/peeb_plots.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5088107d67670278"}},{"code_sha256_prefix":"363a6138162c638c","entry":"get_exp","repo":"anguyen8/peeb","repo_kind":"official","path":"src/misc/peeb_plots.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/misc/peeb_plots.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"363a6138162c638c"}},{"code_sha256_prefix":"d5fd0ce64f775b84","entry":"get_hierarchy_class_name","repo":"anguyen8/peeb","repo_kind":"official","path":"src/prompt_engineering.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/prompt_engineering.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5fd0ce64f775b84"}},{"code_sha256_prefix":"364cc7215ea1dc81","entry":"get_parent_nodes","repo":"anguyen8/peeb","repo_kind":"official","path":"src/prompt_engineering.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/prompt_engineering.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"364cc7215ea1dc81"}},{"code_sha256_prefix":"6d1cc911c7acc3cc","entry":"num_tokens_from_messages","repo":"anguyen8/peeb","repo_kind":"official","path":"src/chatgpt.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/chatgpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d1cc911c7acc3cc"}},{"code_sha256_prefix":"b10b022a231102b0","entry":"test_chatgpt","repo":"anguyen8/peeb","repo_kind":"official","path":"src/chatgpt.py","file_url":"https://github.com/anguyen8/peeb/blob/HEAD/src/chatgpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b10b022a231102b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}