{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/geoclip-clip-inspired-alignment-between","title":"GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization","arxiv_id":"2309.16020","date":"2023-09-27","proceeding":"NeurIPS 2023 11","authors":[],"abstract":"Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. This task has considerable challenges due to immense variation in geographic landscapes. The image-to-image retrieval-based approaches fail to solve this problem on a global scale as it is not feasible to construct a large gallery of images covering the entire world. Instead, existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. However, their performance is limited by the predefined classes and often results in inaccurate localizations when an image's location significantly deviates from its class center. To overcome these limitations, we propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations. GeoCLIP's location encoder models the Earth as a continuous function by employing positional encoding through random Fourier features and constructing a hierarchical representation that captures information at varying resolutions to yield a semantically rich high-dimensional feature suitable to use even beyond geo-localization. To the best of our knowledge, this is the first work employing GPS encoding for geo-localization. We demonstrate the efficacy of our method via extensive experiments and ablations on benchmark datasets. We achieve competitive performance with just 20% of training data, highlighting its effectiveness even in limited-data settings. Furthermore, we qualitatively demonstrate geo-localization using a text query by leveraging CLIP backbone of our image encoder. The project webpage is available at: https://vicentevivan.github.io/GeoCLIP","url_abs":"https://arxiv.org/abs/2309.16020v2","url_pdf":"https://arxiv.org/pdf/2309.16020v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"geoclip-clip-inspired-alignment-between","repo_url":"https://github.com/VicenteVivan/geo-clip","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"geoclip-clip-inspired-alignment-between","repo_url":"https://github.com/kelsdoerksen/giga-connectivity","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"geoclip-clip-inspired-alignment-between","repo_url":"https://github.com/ramanakshay/hierarchical-geoclip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"gps-embeddings","task_name":"GPS Embeddings"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"photo-geolocation-estimation","task_name":"Photo geolocation estimation"},{"task_slug":"visual-place-recognition","task_name":"Visual Place Recognition"},{"task_slug":"geo-localization","task_name":"geo-localization"}],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"gps","method_name":"GPS"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/gps-embeddings-on-geo-tagged-nus-wide-gps-1","task":"GPS Embeddings","dataset":"Geo-Tagged NUS-WIDE (GPS + Visual)","model":"GeoCLIP","rank_in_archive_order":1,"of":8,"metrics":{"mAP":"0.362"},"uses_additional_data":false},{"leaderboard":"/sota/gps-embeddings-on-geo-tagged-nus-wide-gps","task":"GPS Embeddings","dataset":"Geo-Tagged NUS-WIDE (GPS Only)","model":"GeoCLIP","rank_in_archive_order":1,"of":7,"metrics":{" mAP":"0.249"},"uses_additional_data":false},{"leaderboard":"/sota/photo-geolocation-estimation-on-gws15k","task":"Photo geolocation estimation","dataset":"GWS15k","model":"GeoCLIP","rank_in_archive_order":3,"of":5,"metrics":{"City level (25 km)":"3.1","Continent level (2500 km)":"74.1","Country level (750 km)":"45.7","Region level (200 km)":"16.9","Street level (1 km)":"0.6"},"uses_additional_data":false},{"leaderboard":"/sota/photo-geolocation-estimation-on-im2gps3k","task":"Photo geolocation estimation","dataset":"Im2GPS3k","model":"GeoCLIP","rank_in_archive_order":3,"of":14,"metrics":{"City level (25 km)":"34.5","Continent level (2500 km)":"83.8","Country level (750 km)":"69.7","Region level (200 km)":"50.7","Street level (1 km)":"14.1","Training Images":"4.7M"},"uses_additional_data":false},{"leaderboard":"/sota/photo-geolocation-estimation-on-yfcc26k","task":"Photo geolocation estimation","dataset":"YFCC26k","model":"GeoCLIP","rank_in_archive_order":1,"of":6,"metrics":{"City level (25 km)":"22.2","Continent level (2500 km)":"76.0","Country level (750 km)":"57.5","Region level (200 km)":"36.7","Street level (1 km)":"11.6","Training Images":"4.7M"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.16020","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.16020"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kelsdoerksen/giga-connectivity","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VicenteVivan/geo-clip","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ramanakshay/hierarchical-geoclip","reach":null}],"summary":{"ran":2,"ran_honours":3,"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1},"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0342fbed5f4dd6e2","entry":"ImageEncoder","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0342fbed5f4dd6e2"}},{"code_sha256_prefix":"10336c7c42408bd3","entry":"LocationEncoderCapsule","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10336c7c42408bd3"}},{"code_sha256_prefix":"193ffaf24c16bd61","entry":"_extract_image_features","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"193ffaf24c16bd61"}},{"code_sha256_prefix":"0420b6ffe30c1b1b","entry":"equal_earth_projection","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0420b6ffe30c1b1b"}},{"code_sha256_prefix":"59757f850bed5fc9","entry":"load_gps_data","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59757f850bed5fc9"}},{"code_sha256_prefix":"88bf24771293c9e0","entry":"GaussianEncoding","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88bf24771293c9e0"}},{"code_sha256_prefix":"ae05a9ade5fb928f","entry":"GaussianEncoding","repo":"ramanakshay/hierarchical-geoclip","repo_kind":"listed","path":"geoclip/model/geoclip.py","file_url":"https://github.com/ramanakshay/hierarchical-geoclip/blob/HEAD/geoclip/model/geoclip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae05a9ade5fb928f"}},{"code_sha256_prefix":"0db29c57e5099ec7","entry":"GeoCLIP","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0db29c57e5099ec7"}},{"code_sha256_prefix":"7e8b0eadc545633d","entry":"GeoCLIP","repo":"ramanakshay/hierarchical-geoclip","repo_kind":"listed","path":"geoclip/model/geoclip.py","file_url":"https://github.com/ramanakshay/hierarchical-geoclip/blob/HEAD/geoclip/model/geoclip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e8b0eadc545633d"}},{"code_sha256_prefix":"8f9ebe49ed62b249","entry":"ImageEncoder","repo":"ramanakshay/hierarchical-geoclip","repo_kind":"listed","path":"geoclip/model/geoclip.py","file_url":"https://github.com/ramanakshay/hierarchical-geoclip/blob/HEAD/geoclip/model/geoclip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f9ebe49ed62b249"}},{"code_sha256_prefix":"ccb9f7466f37469c","entry":"LocationEncoder","repo":"VicenteVivan/geo-clip","repo_kind":"official","path":"geoclip/model/GeoCLIP.py","file_url":"https://github.com/VicenteVivan/geo-clip/blob/HEAD/geoclip/model/GeoCLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ccb9f7466f37469c"}},{"code_sha256_prefix":"80221b8324d17fd9","entry":"LocationEncoder","repo":"ramanakshay/hierarchical-geoclip","repo_kind":"listed","path":"geoclip/model/geoclip.py","file_url":"https://github.com/ramanakshay/hierarchical-geoclip/blob/HEAD/geoclip/model/geoclip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80221b8324d17fd9"}},{"code_sha256_prefix":"57f6ab5e0536c43d","entry":"LocationEncoderCapsule","repo":"ramanakshay/hierarchical-geoclip","repo_kind":"listed","path":"geoclip/model/geoclip.py","file_url":"https://github.com/ramanakshay/hierarchical-geoclip/blob/HEAD/geoclip/model/geoclip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57f6ab5e0536c43d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}