{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/enhancing-remote-sensing-vision-language","title":"Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification","arxiv_id":"2409.00698","date":"2024-09-01","proceeding":null,"authors":["Karim El Khoury","Maxime Zanella","Benoît Gérin","Tiffanie Godelaine","Benoît Macq","Saïd Mahmoudi","Christophe De Vleeschouwer","Ismail Ben Ayed"],"abstract":"Vision-Language Models for remote sensing have shown promising uses thanks to their extensive pretraining. However, their conventional usage in zero-shot scene classification methods still involves dividing large images into patches and making independent predictions, i.e., inductive inference, thereby limiting their effectiveness by ignoring valuable contextual information. Our approach tackles this issue by utilizing initial predictions based on text prompting and patch affinity relationships from the image encoder to enhance zero-shot capabilities through transductive inference, all without the need for supervision and at a minor computational cost. Experiments on 10 remote sensing datasets with state-of-the-art Vision-Language Models demonstrate significant accuracy improvements over inductive zero-shot classification. Our source code is publicly available on Github: https://github.com/elkhouryk/RS-TransCLIP","url_abs":"https://arxiv.org/abs/2409.00698v2","url_pdf":"https://arxiv.org/pdf/2409.00698v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"enhancing-remote-sensing-vision-language","repo_url":"https://github.com/elkhouryk/rs-transclip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"scene-classification","task_name":"Scene Classification"},{"task_slug":"transductive-zero-shot-classification","task_name":"Transductive Zero-Shot Classification"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":null,"task_name":"zero-shot-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/transductive-zero-shot-classification-on-aid","task":"Transductive Zero-Shot Classification","dataset":"AID","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"92.7"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-1","task":"Transductive Zero-Shot Classification","dataset":"EuroSAT","model":"RS-TransCLIP","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"91.2"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-11","task":"Transductive Zero-Shot Classification","dataset":"MLRSNet","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"78.1"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-12","task":"Transductive Zero-Shot Classification","dataset":"OPTIMAL31","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"94.5"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-13","task":"Transductive Zero-Shot Classification","dataset":"PatternNet","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"96.2"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-14","task":"Transductive Zero-Shot Classification","dataset":"RESISC45","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"88"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-10","task":"Transductive Zero-Shot Classification","dataset":"RSC11","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"88.1"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-15","task":"Transductive Zero-Shot Classification","dataset":"RSICB128","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"54.8"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-16","task":"Transductive Zero-Shot Classification","dataset":"RSICB256","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"72.8"},"uses_additional_data":false},{"leaderboard":"/sota/transductive-zero-shot-classification-on-17","task":"Transductive Zero-Shot Classification","dataset":"WHURS19","model":"RS-TransCLIP","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"99.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.00698","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}