{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusionblocks-blockwise-training-for","title":"DiffusionBlocks: Blockwise Training for Generative Models via Score-Based Diffusion","arxiv_id":"2506.14202","date":"2025-06-17","proceeding":null,"authors":["Makoto Shing","Takuya Akiba"],"abstract":"Training large neural networks with end-to-end backpropagation creates significant memory bottlenecks, limiting accessibility to state-of-the-art AI research. We propose $\\textit{DiffusionBlocks}$, a novel training framework that interprets neural network blocks as performing denoising operations in a continuous-time diffusion process. By partitioning the network into independently trainable blocks and optimizing noise level assignments based on equal cumulative probability mass, our approach achieves significant memory efficiency while maintaining competitive performance compared to traditional backpropagation in generative tasks. Experiments on image generation and language modeling tasks demonstrate memory reduction proportional to the number of blocks while achieving superior performance. DiffusionBlocks provides a promising pathway for democratizing access to large-scale neural network training with limited computational resources.","url_abs":"https://arxiv.org/abs/2506.14202v1","url_pdf":"https://arxiv.org/pdf/2506.14202v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.14202","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.14202"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SakanaAI/DiffusionBlocks","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"vit.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/vit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"fca42c285a2c563a","entry":"get_block_sigmas","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"dblock_modules.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/dblock_modules.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fca42c285a2c563a"}},{"code_sha256_prefix":"767f84ad6a10f380","entry":"get_discrete_sigmas","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"dblock_modules.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/dblock_modules.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"767f84ad6a10f380"}},{"code_sha256_prefix":"ab3ccd3f233dceed","entry":"load_data","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"data.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab3ccd3f233dceed"}},{"code_sha256_prefix":"5c2cfa4f64a9879a","entry":"load_vit","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"vit.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5c2cfa4f64a9879a"}},{"code_sha256_prefix":"e0908de795221aa3","entry":"transforms","repo":"SakanaAI/DiffusionBlocks","repo_kind":"found_in_text","path":"data.py","file_url":"https://github.com/SakanaAI/DiffusionBlocks/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0908de795221aa3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}