{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusionsat-a-generative-foundation-model","title":"DiffusionSat: A Generative Foundation Model for Satellite Imagery","arxiv_id":"2312.03606","date":"2023-12-06","proceeding":null,"authors":["Samar Khanna","Patrick Liu","Linqi Zhou","Chenlin Meng","Robin Rombach","Marshall Burke","David Lobell","Stefano Ermon"],"abstract":"Diffusion models have achieved state-of-the-art results on many modalities including images, speech, and video. However, existing models are not tailored to support remote sensing data, which is widely used in important applications including environmental monitoring and crop-yield prediction. Satellite images are significantly different from natural images -- they can be multi-spectral, irregularly sampled across time -- and existing diffusion models trained on images from the Web do not support them. Furthermore, remote sensing data is inherently spatio-temporal, requiring conditional generation tasks not supported by traditional methods based on captions or images. In this paper, we present DiffusionSat, to date the largest generative foundation model trained on a collection of publicly available large, high-resolution remote sensing datasets. As text-based captions are sparsely available for satellite images, we incorporate the associated metadata such as geolocation as conditioning information. Our method produces realistic samples and can be used to solve multiple generative tasks including temporal generation, superresolution given multi-spectral inputs and in-painting. Our method outperforms previous state-of-the-art methods for satellite image generation and is the first large-scale generative foundation model for satellite imagery. The project website can be found here: https://samar-khanna.github.io/DiffusionSat/","url_abs":"https://arxiv.org/abs/2312.03606v2","url_pdf":"https://arxiv.org/pdf/2312.03606v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusionsat-a-generative-foundation-model","repo_url":"https://github.com/samar-khanna/diffusionsat","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"crop-yield-prediction","task_name":"Crop Yield Prediction"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.03606","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.03606"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samar-khanna/diffusionsat","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"unverified":5},"by_repo_kind":{"listed":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bd5a05ce886da4ce","entry":"image_grid","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"train_controlnet.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/train_controlnet.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bd5a05ce886da4ce"}},{"code_sha256_prefix":"e82aa0b890d4b334","entry":"fmow_tokenize_caption","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"diffusionsat/data_util.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/diffusionsat/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e82aa0b890d4b334"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"train_controlnet.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/train_controlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}},{"code_sha256_prefix":"31a9bdfa0b207f42","entry":"mercator_to_geo","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"diffusionsat/satlas_util.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/diffusionsat/satlas_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31a9bdfa0b207f42"}},{"code_sha256_prefix":"48bb4b20422cc224","entry":"satlas_tokenize_caption","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"diffusionsat/data_util.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/diffusionsat/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48bb4b20422cc224"}},{"code_sha256_prefix":"da94debb8019ad46","entry":"zero_module","repo":"samar-khanna/diffusionsat","repo_kind":"listed","path":"diffusionsat/controlnet.py","file_url":"https://github.com/samar-khanna/diffusionsat/blob/HEAD/diffusionsat/controlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da94debb8019ad46"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}