{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mask-is-all-you-need","title":"[MASK] is All You Need","arxiv_id":"2412.06787","date":"2024-12-09","proceeding":null,"authors":["Vincent Tao Hu","Björn Ommer"],"abstract":"In generative models, two paradigms have gained attraction in various applications: next-set prediction-based Masked Generative Models and next-noise prediction-based Non-Autoregressive Models, e.g., Diffusion Models. In this work, we propose using discrete-state models to connect them and explore their scalability in the vision domain. First, we conduct a step-by-step analysis in a unified design space across two types of models including timestep-independence, noise schedule, temperature, guidance strength, etc in a scalable manner. Second, we re-cast typical discriminative tasks, e.g., image segmentation, as an unmasking process from [MASK] tokens on a discrete-state model. This enables us to perform various sampling processes, including flexible conditional sampling by only training once to model the joint distribution. All aforementioned explorations lead to our framework named Discrete Interpolants, which enables us to achieve state-of-the-art or competitive performance compared to previous discrete-state based methods in various benchmarks, like ImageNet256, MS COCO, and video dataset FaceForensics. In summary, by leveraging [MASK] in discrete-state models, we can bridge Masked Generative and Non-autoregressive Diffusion models, as well as generative and discriminative tasks.","url_abs":"https://arxiv.org/abs/2412.06787v2","url_pdf":"https://arxiv.org/pdf/2412.06787v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mask-is-all-you-need","repo_url":"https://github.com/CompVis/mask","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.06787","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.06787"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CompVis/mask","reach":null}],"summary":{"ran_draft_wrong":4,"ran_fixture":3,"ran_violates":1,"ran":1,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"834bbad6b027b869","entry":"custom_to_np","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"834bbad6b027b869"}},{"code_sha256_prefix":"73fba7367428c8a5","entry":"custom_to_pil","repo":"CompVis/mask","repo_kind":"official","path":"ldm/scripts/sample_diffusion.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/ldm/scripts/sample_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73fba7367428c8a5"}},{"code_sha256_prefix":"137bf88ab8750134","entry":"extend_as","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"137bf88ab8750134"}},{"code_sha256_prefix":"70517741ca818319","entry":"indices_to_diracp","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"70517741ca818319"}},{"code_sha256_prefix":"968eb3375c1d67e1","entry":"logs2pil","repo":"CompVis/mask","repo_kind":"official","path":"ldm/scripts/sample_diffusion.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/ldm/scripts/sample_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"968eb3375c1d67e1"}},{"code_sha256_prefix":"650ed748a9500619","entry":"sample_p","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"650ed748a9500619"}},{"code_sha256_prefix":"7629721cb0905b41","entry":"search_ah","repo":"CompVis/mask","repo_kind":"official","path":"ldm/scripts/train_searcher.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/ldm/scripts/train_searcher.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7629721cb0905b41"}},{"code_sha256_prefix":"872823964fff640d","entry":"search_bruteforce","repo":"CompVis/mask","repo_kind":"official","path":"ldm/scripts/train_searcher.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/ldm/scripts/train_searcher.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"872823964fff640d"}},{"code_sha256_prefix":"eda17c2c32cd3500","entry":"search_partioned_ah","repo":"CompVis/mask","repo_kind":"official","path":"ldm/scripts/train_searcher.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/ldm/scripts/train_searcher.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eda17c2c32cd3500"}},{"code_sha256_prefix":"04cecb2affe0b210","entry":"Coupling","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04cecb2affe0b210"}},{"code_sha256_prefix":"a69c41311ff4819e","entry":"DiscreteInterpolants","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a69c41311ff4819e"}},{"code_sha256_prefix":"8ed8619511cec220","entry":"KappaScheduler","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8ed8619511cec220"}},{"code_sha256_prefix":"e927bede2e52cbc7","entry":"logits_with_top_k_top_p_","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e927bede2e52cbc7"}},{"code_sha256_prefix":"a41a92fc83ae68c8","entry":"print_rank_0","repo":"CompVis/mask","repo_kind":"official","path":"dynamics/dynamic_di.py","file_url":"https://github.com/CompVis/mask/blob/HEAD/dynamics/dynamic_di.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a41a92fc83ae68c8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}