{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zoomldm-latent-diffusion-model-for-multi","title":"ZoomLDM: Latent Diffusion Model for multi-scale image generation","arxiv_id":"2411.16969","date":"2024-11-25","proceeding":"CVPR 2025 1","authors":["Srikar Yellapragada","Alexandros Graikos","Kostas Triaridis","Prateek Prasanna","Rajarsi R. Gupta","Joel Saltz","Dimitris Samaras"],"abstract":"Diffusion models have revolutionized image generation, yet several challenges restrict their application to large-image domains, such as digital pathology and satellite imagery. Given that it is infeasible to directly train a model on 'whole' images from domains with potential gigapixel sizes, diffusion-based generative methods have focused on synthesizing small, fixed-size patches extracted from these images. However, generating small patches has limited applicability since patch-based models fail to capture the global structures and wider context of large images, which can be crucial for synthesizing (semantically) accurate samples. In this paper, to overcome this limitation, we present ZoomLDM, a diffusion model tailored for generating images across multiple scales. Central to our approach is a novel magnification-aware conditioning mechanism that utilizes self-supervised learning (SSL) embeddings and allows the diffusion model to synthesize images at different 'zoom' levels, i.e., fixed-size patches extracted from large images at varying scales. ZoomLDM achieves state-of-the-art image generation quality across all scales, excelling particularly in the data-scarce setting of generating thumbnails of entire large images. The multi-scale nature of ZoomLDM unlocks additional capabilities in large image generation, enabling computationally tractable and globally coherent image synthesis up to $4096 \\times 4096$ pixels and $4\\times$ super-resolution. Additionally, multi-scale features extracted from ZoomLDM are highly effective in multiple instance learning experiments. We provide high-resolution examples of the generated images on our website https://histodiffusion.github.io/docs/publications/zoomldm/.","url_abs":"https://arxiv.org/abs/2411.16969v1","url_pdf":"https://arxiv.org/pdf/2411.16969v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zoomldm-latent-diffusion-model-for-multi","repo_url":"https://github.com/cvlab-stonybrook/ZoomLDM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"multiple-instance-learning","task_name":"Multiple Instance Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.16969","atlas_url":"https://app.syntology.ai/?focus=2411.16969","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.16969"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cvlab-stonybrook/ZoomLDM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_draft_wrong":3,"ran_violates":2,"ran_fixture":1,"unverified":10},"by_repo_kind":{"official":{"samples":20,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"372b275922a9f531","entry":"box","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/resizer.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/resizer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"372b275922a9f531"}},{"code_sha256_prefix":"b2cc78f4103df806","entry":"create_logger","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"cdm_dit/train_cdm.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/cdm_dit/train_cdm.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2cc78f4103df806"}},{"code_sha256_prefix":"ee77cd8fde21ffd1","entry":"cubic","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/resizer.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/resizer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee77cd8fde21ffd1"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"e5947aba1d10885f","entry":"get_1d_sincos_pos_embed_from_grid","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"cdm_dit/models.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/cdm_dit/models.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5947aba1d10885f"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"f106699fc610a684","entry":"lanczos2","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/resizer.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/resizer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f106699fc610a684"}},{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"cdm_dit/models.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/cdm_dit/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"32af1c12fd826ef2","entry":"autocast","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32af1c12fd826ef2"}},{"code_sha256_prefix":"97c840d12ce5cf07","entry":"collate_fn","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"utils.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97c840d12ce5cf07"}},{"code_sha256_prefix":"8163165d266649ff","entry":"decode_large_image","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/utils.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8163165d266649ff"}},{"code_sha256_prefix":"cd927cf05827a83d","entry":"gaussian_kernel","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/utils.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd927cf05827a83d"}},{"code_sha256_prefix":"ec487ba24ac66507","entry":"get_conditioning","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/postprocess.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/postprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec487ba24ac66507"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}},{"code_sha256_prefix":"a37c1537ad066a85","entry":"model_pred","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/utils.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a37c1537ad066a85"}},{"code_sha256_prefix":"865a41ede071b2cb","entry":"my_collate","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"cdm_dit/train_cdm.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/cdm_dit/train_cdm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"865a41ede071b2cb"}},{"code_sha256_prefix":"34bbbbc4a2ae919a","entry":"postprocess_image","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/postprocess.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/postprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34bbbbc4a2ae919a"}},{"code_sha256_prefix":"ecc1c08d5ab92766","entry":"slerp","repo":"cvlab-stonybrook/ZoomLDM","repo_kind":"official","path":"large_image_gen/postprocess.py","file_url":"https://github.com/cvlab-stonybrook/ZoomLDM/blob/HEAD/large_image_gen/postprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecc1c08d5ab92766"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}