{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/all-atom-diffusion-transformers-unified","title":"All-atom Diffusion Transformers: Unified generative modelling of molecules and materials","arxiv_id":"2503.03965","date":"2025-03-05","proceeding":null,"authors":["Chaitanya K. Joshi","Xiang Fu","Yi-Lun Liao","Vahe Gharakhanyan","Benjamin Kurt Miller","Anuroop Sriram","Zachary W. Ulissi"],"abstract":"Diffusion models are the standard toolkit for generative modelling of 3D atomic systems. However, for different types of atomic systems - such as molecules and materials - the generative processes are usually highly specific to the target system despite the underlying physics being the same. We introduce the All-atom Diffusion Transformer (ADiT), a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems using the same model: (1) An autoencoder maps a unified, all-atom representations of molecules and materials to a shared latent embedding space; and (2) A diffusion model is trained to generate new latent embeddings that the autoencoder can decode to sample new molecules or materials. Experiments on QM9 and MP20 datasets demonstrate that jointly trained ADiT generates realistic and valid molecules as well as materials, exceeding state-of-the-art results from molecule and crystal-specific models. ADiT uses standard Transformers for both the autoencoder and diffusion model, resulting in significant speedups during training and inference compared to equivariant diffusion models. Scaling ADiT up to half a billion parameters predictably improves performance, representing a step towards broadly generalizable foundation models for generative chemistry. Open source code: https://github.com/facebookresearch/all-atom-diffusion-transformer","url_abs":"https://arxiv.org/abs/2503.03965v1","url_pdf":"https://arxiv.org/pdf/2503.03965v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"all-atom-diffusion-transformers-unified","repo_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"unconditional-crystal-generation","task_name":"Unconditional Crystal Generation"},{"task_slug":"unconditional-molecule-generation","task_name":"Unconditional Molecule Generation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/unconditional-crystal-generation-on-mp20","task":"Unconditional Crystal Generation","dataset":"MP20","model":"ADiT","rank_in_archive_order":1,"of":3,"metrics":{"DFT Stable, Unique, Novel Rate":"6.0","Validity":"91.92"},"uses_additional_data":false},{"leaderboard":"/sota/unconditional-molecule-generation-on-geom","task":"Unconditional Molecule Generation","dataset":"GEOM-DRUGS","model":"ADiT","rank_in_archive_order":3,"of":5,"metrics":{"PoseBusters Atoms Connected":"93.0","PoseBusters Validity":"85.3","Validity":"95.3"},"uses_additional_data":false},{"leaderboard":"/sota/unconditional-molecule-generation-on-qm9","task":"Unconditional Molecule Generation","dataset":"QM9","model":"ADiT","rank_in_archive_order":1,"of":4,"metrics":{"PoseBusters Internal Energy":"95.86","Validity":"94.45"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.03965","atlas_url":"https://app.syntology.ai/?focus=2503.03965","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.03965"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/all-atom-diffusion-transformer","reach":null}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"3e2024b54c710632","entry":"DiTBlock","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3e2024b54c710632"}},{"code_sha256_prefix":"9e3ed34320e0b553","entry":"FinalLayer","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9e3ed34320e0b553"}},{"code_sha256_prefix":"d712ae7c7b20e719","entry":"LabelEmbedder","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d712ae7c7b20e719"}},{"code_sha256_prefix":"0694baedef001ec2","entry":"Mlp","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0694baedef001ec2"}},{"code_sha256_prefix":"4f5374daa55652e7","entry":"TimestepEmbedder","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4f5374daa55652e7"}},{"code_sha256_prefix":"792925249b797c34","entry":"get_pos_embedding","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"792925249b797c34"}},{"code_sha256_prefix":"739908da4069e104","entry":"DiT","repo":"facebookresearch/all-atom-diffusion-transformer","repo_kind":"official","path":"src/models/denoisers/dit.py","file_url":"https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/HEAD/src/models/denoisers/dit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"739908da4069e104"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}