{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unisolver-pde-conditional-transformers-are","title":"Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers","arxiv_id":"2405.17527","date":"2024-05-27","proceeding":null,"authors":["Hang Zhou","Yuezhou Ma","Haixu Wu","Haowen Wang","Mingsheng Long"],"abstract":"Deep models have recently emerged as a promising tool to solve partial differential equations (PDEs), known as neural PDE solvers. While neural solvers trained from either simulation data or physics-informed loss can solve PDEs reasonably well, they are mainly restricted to a few instances of PDEs, e.g. a certain equation with a limited set of coefficients. This limits the generalization of neural solvers to diverse PDEs, impeding them from being practical surrogate models for numerical solvers. In this paper, we present the Universal PDE Solver (Unisolver) capable of solving a wide scope of PDEs by training a novel Transformer model on diverse data and conditioned on diverse PDEs. Instead of purely scaling up data and parameters, Unisolver stems from the theoretical analysis of the PDE-solving process. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components, e.g. equation symbols, coefficients, and boundary conditions. Inspired by the mathematical structure of PDEs, we define a complete set of PDE components and flexibly embed them as domain-wise (e.g. equation symbols) and point-wise (e.g. boundaries) conditions for Transformer PDE solvers. Integrating physical insights with recent Transformer advances, Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks, showing impressive performance gains and favorable PDE generalizability.","url_abs":"https://arxiv.org/abs/2405.17527v3","url_pdf":"https://arxiv.org/pdf/2405.17527v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unisolver-pde-conditional-transformers-are","repo_url":"https://github.com/thuml/Unisolver","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"set","method_name":"SET"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.17527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.17527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thuml/Unisolver","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_honours":1,"ran_violates":1,"unverified":3},"by_repo_kind":{"listed":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ff22c81e9f140b90","entry":"count_params","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/utilities3.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/utilities3.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff22c81e9f140b90"}},{"code_sha256_prefix":"62fcb3912a967a50","entry":"modulate","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/models/Unisolver_1D.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/models/Unisolver_1D.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62fcb3912a967a50"}},{"code_sha256_prefix":"6ba8cee9f5daea41","entry":"pair","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/models/Unisolver_1D.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/models/Unisolver_1D.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ba8cee9f5daea41"}},{"code_sha256_prefix":"02f4382376194959","entry":"get_llm_embeddings","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/exp_1D_PDEs.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/exp_1D_PDEs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02f4382376194959"}},{"code_sha256_prefix":"f49c0602f026df64","entry":"modulate_inr","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/models/Unisolver_1D.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/models/Unisolver_1D.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f49c0602f026df64"}},{"code_sha256_prefix":"07d698a590663c2d","entry":"pdist","repo":"thuml/Unisolver","repo_kind":"listed","path":"1D_Time_Dependent_PDE/utilities3.py","file_url":"https://github.com/thuml/Unisolver/blob/HEAD/1D_Time_Dependent_PDE/utilities3.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07d698a590663c2d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}