{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/masked-diffusion-transformer-is-a-strong","title":"MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer","arxiv_id":"2303.14389","date":"2023-03-25","proceeding":"ICCV 2023 1","authors":["ShangHua Gao","Pan Zhou","Ming-Ming Cheng","Shuicheng Yan"],"abstract":"Despite its success in image synthesis, we observe that diffusion probabilistic models (DPMs) often lack contextual reasoning ability to learn the relations among object parts in an image, leading to a slow learning process. To solve this issue, we propose a Masked Diffusion Transformer (MDT) that introduces a mask latent modeling scheme to explicitly enhance the DPMs' ability to contextual relation learning among object semantic parts in an image. During training, MDT operates in the latent space to mask certain tokens. Then, an asymmetric diffusion transformer is designed to predict masked tokens from unmasked ones while maintaining the diffusion generation process. Our MDT can reconstruct the full information of an image from its incomplete contextual input, thus enabling it to learn the associated relations among image tokens. We further improve MDT with a more efficient macro network structure and training strategy, named MDTv2. Experimental results show that MDTv2 achieves superior image synthesis performance, e.g., a new SOTA FID score of 1.58 on the ImageNet dataset, and has more than 10x faster learning speed than the previous SOTA DiT. The source code is released at https://github.com/sail-sg/MDT.","url_abs":"https://arxiv.org/abs/2303.14389v2","url_pdf":"https://arxiv.org/pdf/2303.14389v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"masked-diffusion-transformer-is-a-strong","repo_url":"https://github.com/sail-sg/mdt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-imagenet-256x256","task":"Image Generation","dataset":"ImageNet 256x256","model":"MDTv2","rank_in_archive_order":27,"of":94,"metrics":{"FID":"1.58"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-256x256","task":"Image Generation","dataset":"ImageNet 256x256","model":"MDT","rank_in_archive_order":40,"of":94,"metrics":{"FID":"1.79"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.14389","atlas_url":"https://app.syntology.ai/?focus=2303.14389","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.14389"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sail-sg/mdt","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/sail-sg/MDT","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"07d39f4cc0b2a473","entry":"Attention","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07d39f4cc0b2a473"}},{"code_sha256_prefix":"b6005b513a46b17e","entry":"FinalLayer","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6005b513a46b17e"}},{"code_sha256_prefix":"e10926ae02924e62","entry":"LabelEmbedder","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e10926ae02924e62"}},{"code_sha256_prefix":"f1228667bd65606d","entry":"RelativePositionBias","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f1228667bd65606d"}},{"code_sha256_prefix":"56d0dbde01f5df9a","entry":"MDTBlock","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56d0dbde01f5df9a"}},{"code_sha256_prefix":"90de145f5347518d","entry":"MDTv2","repo":"sail-sg/MDT","repo_kind":"official","path":"masked_diffusion/models.py","file_url":"https://github.com/sail-sg/MDT/blob/HEAD/masked_diffusion/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90de145f5347518d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}