{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/chatgpt-powered-hierarchical-comparisons-for","title":"ChatGPT-Powered Hierarchical Comparisons for Image Classification","arxiv_id":null,"date":"2023-09-21","proceeding":"NeurIPS 2023 11","authors":[],"abstract":"The zero-shot open-vocabulary setting poses challenges for image classification.\nFortunately, utilizing a vision-language model like CLIP, pre-trained on image-text\npairs, allows for classifying images by comparing embeddings. Leveraging large\nlanguage models (LLMs) such as ChatGPT can further enhance CLIP’s accuracy\nby incorporating class-specific knowledge in descriptions. However, CLIP still\nexhibits a bias towards certain classes and generates similar descriptions for similar\nclasses, disregarding their differences. To address this problem, we present a\nnovel image classification framework via hierarchical comparisons. By recursively\ncomparing and grouping classes with LLMs, we construct a class hierarchy. With\nsuch a hierarchy, we can classify an image by descending from the top to the bottom\nof the hierarchy, comparing image and text embeddings at each level. Through\nextensive experiments and analyses, we demonstrate that our proposed approach is\nintuitive, effective, and explainable. Code will be released upon publication.","url_abs":"https://openreview.net/forum?id=GTYaYNsFyv","url_pdf":"https://openreview.net/pdf?id=GTYaYNsFyv","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"chatgpt-powered-hierarchical-comparisons-for","repo_url":"https://github.com/zhiyuan-r/chatgpt-powered-hierarchical-comparisons-for-image-classification","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}