{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-diffusion-models-with-state-space","title":"Scalable Diffusion Models with State Space Backbone","arxiv_id":"2402.05608","date":"2024-02-08","proceeding":null,"authors":["Zhengcong Fei","Mingyuan Fan","Changqian Yu","Junshi Huang"],"abstract":"This paper presents a new exploration into a category of diffusion models built upon state space architecture. We endeavor to train diffusion models for image data, wherein the traditional U-Net backbone is supplanted by a state space backbone, functioning on raw patches or latent space. Given its notable efficacy in accommodating long-range dependencies, Diffusion State Space Models (DiS) are distinguished by treating all inputs including time, condition, and noisy image patches as tokens. Our assessment of DiS encompasses both unconditional and class-conditional image generation scenarios, revealing that DiS exhibits comparable, if not superior, performance to CNN-based or Transformer-based U-Net architectures of commensurate size. Furthermore, we analyze the scalability of DiS, gauged by the forward pass complexity quantified in Gflops. DiS models with higher Gflops, achieved through augmentation of depth/width or augmentation of input tokens, consistently demonstrate lower FID. In addition to demonstrating commendable scalability characteristics, DiS-H/2 models in latent space achieve performance levels akin to prior diffusion models on class-conditional ImageNet benchmarks at the resolution of 256$\\times$256 and 512$\\times$512, while significantly reducing the computational burden. The code and models are available at: https://github.com/feizc/DiS.","url_abs":"https://arxiv.org/abs/2402.05608v3","url_pdf":"https://arxiv.org/pdf/2402.05608v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-diffusion-models-with-state-space","repo_url":"https://github.com/feizc/dis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"scalable-diffusion-models-with-state-space","repo_url":"https://github.com/hustvl/dig","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.05608","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.05608"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/feizc/dis","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hustvl/dig","reach":null}],"summary":{"ran_draft_wrong":5,"ran":1,"ran_honours":3,"ran_fixture":3,"unverified":1},"by_repo_kind":{"official":{"samples":13,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"d6a68e210556f857","entry":"approx_standard_normal_cdf","repo":"feizc/dis","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6a68e210556f857"}},{"code_sha256_prefix":"0595d8b8521fd9ca","entry":"causal_conv1d_ref","repo":"feizc/dis","repo_kind":"official","path":"causal-conv1d/causal_conv1d/causal_conv1d_interface.py","file_url":"https://github.com/feizc/dis/blob/HEAD/causal-conv1d/causal_conv1d/causal_conv1d_interface.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0595d8b8521fd9ca"}},{"code_sha256_prefix":"ab1c9568b4e13899","entry":"continuous_gaussian_log_likelihood","repo":"feizc/dis","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab1c9568b4e13899"}},{"code_sha256_prefix":"3e0fa4efc22272d4","entry":"get_beta_schedule","repo":"feizc/dis","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e0fa4efc22272d4"}},{"code_sha256_prefix":"36e30c7fb679ec78","entry":"get_named_beta_schedule","repo":"feizc/dis","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36e30c7fb679ec78"}},{"code_sha256_prefix":"f6d7c009a8efb8b7","entry":"mean_flat","repo":"feizc/dis","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6d7c009a8efb8b7"}},{"code_sha256_prefix":"8afbfc42c6ea0448","entry":"normal_kl","repo":"feizc/dis","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8afbfc42c6ea0448"}},{"code_sha256_prefix":"9cf076a2ae913f94","entry":"patchify","repo":"feizc/dis","repo_kind":"official","path":"models_dis.py","file_url":"https://github.com/feizc/dis/blob/HEAD/models_dis.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9cf076a2ae913f94"}},{"code_sha256_prefix":"ba7b868fec181b57","entry":"selective_scan_ref","repo":"feizc/dis","repo_kind":"official","path":"mamba/mamba_ssm/ops/selective_scan_interface.py","file_url":"https://github.com/feizc/dis/blob/HEAD/mamba/mamba_ssm/ops/selective_scan_interface.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba7b868fec181b57"}},{"code_sha256_prefix":"ea9dbc131adf582e","entry":"space_timesteps","repo":"feizc/dis","repo_kind":"official","path":"diffusion/respace.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/respace.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea9dbc131adf582e"}},{"code_sha256_prefix":"270cd726f1952c2a","entry":"timestep_embedding","repo":"feizc/dis","repo_kind":"official","path":"models_dis.py","file_url":"https://github.com/feizc/dis/blob/HEAD/models_dis.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"270cd726f1952c2a"}},{"code_sha256_prefix":"f9244c08c1011341","entry":"unpatchify","repo":"feizc/dis","repo_kind":"official","path":"models_dis.py","file_url":"https://github.com/feizc/dis/blob/HEAD/models_dis.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9244c08c1011341"}},{"code_sha256_prefix":"e48218d7d73db0b3","entry":"create_named_schedule_sampler","repo":"feizc/dis","repo_kind":"official","path":"diffusion/timestep_sampler.py","file_url":"https://github.com/feizc/dis/blob/HEAD/diffusion/timestep_sampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e48218d7d73db0b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}