{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/g3-an-effective-and-adaptive-framework-for","title":"G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models","arxiv_id":"2405.14702","date":"2024-05-23","proceeding":null,"authors":["Pengyue Jia","Yiding Liu","Xiaopeng Li","Yuhao Wang","Yantong Du","Xiao Han","Xuetao Wei","Shuaiqiang Wang","Dawei Yin","Xiangyu Zhao"],"abstract":"Worldwide geolocalization aims to locate the precise location at the coordinate level of photos taken anywhere on the Earth. It is very challenging due to 1) the difficulty of capturing subtle location-aware visual semantics, and 2) the heterogeneous geographical distribution of image data. As a result, existing studies have clear limitations when scaled to a worldwide context. They may easily confuse distant images with similar visual contents, or cannot adapt to various locations worldwide with different amounts of relevant data. To resolve these limitations, we propose G3, a novel framework based on Retrieval-Augmented Generation (RAG). In particular, G3 consists of three steps, i.e., Geo-alignment, Geo-diversification, and Geo-verification to optimize both retrieval and generation phases of worldwide geolocalization. During Geo-alignment, our solution jointly learns expressive multi-modal representations for images, GPS and textual descriptions, which allows us to capture location-aware semantics for retrieving nearby images for a given query. During Geo-diversification, we leverage a prompt ensembling method that is robust to inconsistent retrieval performance for different image queries. Finally, we combine both retrieved and generated GPS candidates in Geo-verification for location prediction. Experiments on two well-established datasets IM2GPS3k and YFCC4k verify the superiority of G3 compared to other state-of-the-art methods. Our code and data are available online for reproduction.","url_abs":"https://arxiv.org/abs/2405.14702v2","url_pdf":"https://arxiv.org/pdf/2405.14702v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"g3-an-effective-and-adaptive-framework-for","repo_url":"https://github.com/applied-machine-learning-lab/g3","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"photo-geolocation-estimation","task_name":"Photo geolocation estimation"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[{"method_slug":"gps","method_name":"GPS"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/photo-geolocation-estimation-on-im2gps3k","task":"Photo geolocation estimation","dataset":"Im2GPS3k","model":"G3","rank_in_archive_order":2,"of":14,"metrics":{"City level (25 km)":"40.94","Continent level (2500 km)":"84.68","Country level (750 km)":"71.24","Region level (200 km)":"55.56","Street level (1 km)":"16.65","Training Images":"4.12M"},"uses_additional_data":false},{"leaderboard":"/sota/photo-geolocation-estimation-on-yfcc4k","task":"Photo geolocation estimation","dataset":"YFCC4k","model":"G3","rank_in_archive_order":3,"of":4,"metrics":{"City (25 km)":"35.89","Continent (2500 km)":"78.15","Country (750 km)":"64.26","Median Error (km)":"/","Region (200 km)":"46.98","Street (1 km)":"23.99"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.14702","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14702"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/applied-machine-learning-lab/g3","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2d2bc5c34037e15b","entry":"basic_encoding","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"utils/rff/functional.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/utils/rff/functional.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d2bc5c34037e15b"}},{"code_sha256_prefix":"f41cb1a19b154297","entry":"encode_image","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"llm_predict.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/llm_predict.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f41cb1a19b154297"}},{"code_sha256_prefix":"93245a6fbe3c4415","entry":"gaussian_encoding","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"utils/rff/functional.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/utils/rff/functional.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93245a6fbe3c4415"}},{"code_sha256_prefix":"14ab877eaf533dfb","entry":"get_response","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"llm_predict.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/llm_predict.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14ab877eaf533dfb"}},{"code_sha256_prefix":"a32ec97a24a3f7e6","entry":"get_response","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"llm_predict_hf.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/llm_predict_hf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a32ec97a24a3f7e6"}},{"code_sha256_prefix":"96b18a8aed30ff38","entry":"get_response_rag","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"llm_predict.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/llm_predict.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96b18a8aed30ff38"}},{"code_sha256_prefix":"09307a86733e770b","entry":"get_response_rag","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"llm_predict_hf.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/llm_predict_hf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"09307a86733e770b"}},{"code_sha256_prefix":"43e5f269780ac7f5","entry":"sample_b","repo":"applied-machine-learning-lab/g3","repo_kind":"official","path":"utils/rff/functional.py","file_url":"https://github.com/applied-machine-learning-lab/g3/blob/HEAD/utils/rff/functional.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43e5f269780ac7f5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}