Papers › Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
Chen-Hao Chao, Wei-Fang Sun, Hanwen Liang, Chun-Yi Lee, Rahul G. Krishnan
Masked diffusion models (MDM) are powerful generative models for discrete data that generate samples by progressively unmasking tokens in a sequence. Each token can take one of two states: masked or unmasked. We observe that token sequences often remain unchanged between consecutive sampling steps; consequently, the model repeatedly processes identical inputs, leading to redundant computation. To address this inefficiency, we propose the Partial masking scheme (Prime), which augments MDM by allowing tokens to take intermediate states interpolated between the masked and unmasked states. This design enables the model to make predictions based on partially observed token information, and facilitates a fine-grained denoising process. We derive a variational training objective and introduce a simple architectural design to accommodate intermediate-state inputs. Our method demonstrates superior performance across a diverse set of generative modeling tasks. On text data, it achieves a perplexity of 15.36 on OpenWebText, outperforming previous MDM (21.52), autoregressive models (17.54), and their hybrid variants (17.58), without relying on an autoregressive formulation. On image data, it attains competitive FID scores of 3.26 on CIFAR-10 and 6.98 on ImageNet-32, comparable to leading continuous generative models.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image Generation | CIFAR-10 | MDM-Prime | FID | 3.26 | #29 of 78 | Archive leaderboard | report |
| Image Generation | CIFAR-10 | MDM-Prime | IS | 9.67 | #29 of 78 | Archive leaderboard | report |
| Image Generation | CIFAR-10 | MDM | FID | 4.66 | #33 of 78 | Archive leaderboard | report |
| Image Generation | CIFAR-10 | MDM | IS | 9.09 | #33 of 78 | Archive leaderboard | report |
| Image Generation | ImageNet 32x32 | MDM-Prime | FID | 6.98 | #6 of 35 | Archive leaderboard | report |
| Image Generation | ImageNet 32x32 | MDM-Prime | Inception score | 11.65 | #6 of 35 | Archive leaderboard | report |
| Image Generation | ImageNet 32x32 | MDM | FID | 7.91 | #7 of 35 | Archive leaderboard | report |
| Image Generation | ImageNet 32x32 | MDM | Inception score | 11.60 | #7 of 35 | Archive leaderboard | report |
| Language Modelling | OpenWebText | MDLM-Prime | eval_perplexity | 15.36 | #1 of 12 | Archive leaderboard | report |
| Language Modelling | OpenWebText | MDLM-Prime | parameters | 131M | #1 of 12 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections