{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/remix-dit-mixing-diffusion-transformers-for","title":"Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising","arxiv_id":"2412.05628","date":"2024-12-07","proceeding":null,"authors":["Gongfan Fang","Xinyin Ma","Xinchao Wang"],"abstract":"Transformer-based diffusion models have achieved significant advancements across a variety of generative tasks. However, producing high-quality outputs typically necessitates large transformer models, which result in substantial training and inference overhead. In this work, we investigate an alternative approach involving multiple experts for denoising, and introduce Remix-DiT, a novel method designed to enhance output quality at a low cost. The goal of Remix-DiT is to craft N diffusion experts for different denoising timesteps, yet without the need for expensive training of N independent models. To achieve this, Remix-DiT employs K basis models (where K < N) and utilizes learnable mixing coefficients to adaptively craft expert models. This design offers two significant advantages: first, although the total model size is increased, the model produced by the mixing operation shares the same architecture as a plain model, making the overall model as efficient as a standard diffusion transformer. Second, the learnable mixing adaptively allocates model capacity across timesteps, thereby effectively improving generation quality. Experiments conducted on the ImageNet dataset demonstrate that Remix-DiT achieves promising results compared to standard diffusion transformers and other multiple-expert methods. The code is available at https://github.com/VainF/Remix-DiT.","url_abs":"https://arxiv.org/abs/2412.05628v1","url_pdf":"https://arxiv.org/pdf/2412.05628v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"remix-dit-mixing-diffusion-transformers-for","repo_url":"https://github.com/vainf/remix-dit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2412.05628","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.05628"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vainf/remix-dit","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":6,"ran_honours":4,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":16,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"d6a68e210556f857","entry":"approx_standard_normal_cdf","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6a68e210556f857"}},{"code_sha256_prefix":"1712a07966b542ee","entry":"center_crop_arr","repo":"vainf/remix-dit","repo_kind":"official","path":"extract_features.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/extract_features.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1712a07966b542ee"}},{"code_sha256_prefix":"ab1c9568b4e13899","entry":"continuous_gaussian_log_likelihood","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab1c9568b4e13899"}},{"code_sha256_prefix":"7b21a01ae77703a3","entry":"create_npz_from_sample_folder","repo":"vainf/remix-dit","repo_kind":"official","path":"sample_ddp.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/sample_ddp.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b21a01ae77703a3"}},{"code_sha256_prefix":"c92c27c924b517e8","entry":"get_2d_sincos_pos_embed","repo":"vainf/remix-dit","repo_kind":"official","path":"remix_dit.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/remix_dit.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c92c27c924b517e8"}},{"code_sha256_prefix":"3e0fa4efc22272d4","entry":"get_beta_schedule","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e0fa4efc22272d4"}},{"code_sha256_prefix":"36e30c7fb679ec78","entry":"get_named_beta_schedule","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36e30c7fb679ec78"}},{"code_sha256_prefix":"f6d7c009a8efb8b7","entry":"mean_flat","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6d7c009a8efb8b7"}},{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"vainf/remix-dit","repo_kind":"official","path":"remix_dit.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/remix_dit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"8afbfc42c6ea0448","entry":"normal_kl","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8afbfc42c6ea0448"}},{"code_sha256_prefix":"ea9dbc131adf582e","entry":"space_timesteps","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/respace.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/respace.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea9dbc131adf582e"}},{"code_sha256_prefix":"8382f7e48cc8a309","entry":"create_logger","repo":"vainf/remix-dit","repo_kind":"official","path":"extract_features.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/extract_features.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8382f7e48cc8a309"}},{"code_sha256_prefix":"e48218d7d73db0b3","entry":"create_named_schedule_sampler","repo":"vainf/remix-dit","repo_kind":"official","path":"diffusion/timestep_sampler.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/diffusion/timestep_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e48218d7d73db0b3"}},{"code_sha256_prefix":"6a0d5ecd441905cc","entry":"download_model","repo":"vainf/remix-dit","repo_kind":"official","path":"download.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/download.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a0d5ecd441905cc"}},{"code_sha256_prefix":"865b5887184a73f9","entry":"find_model","repo":"vainf/remix-dit","repo_kind":"official","path":"download.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/download.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"865b5887184a73f9"}},{"code_sha256_prefix":"665d8a4e8f673a4c","entry":"get_2d_sincos_pos_embed_from_grid","repo":"vainf/remix-dit","repo_kind":"official","path":"remix_dit.py","file_url":"https://github.com/vainf/remix-dit/blob/HEAD/remix_dit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"665d8a4e8f673a4c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}