{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pinnsformer-a-transformer-based-framework-for","title":"PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks","arxiv_id":"2307.11833","date":"2023-07-21","proceeding":null,"authors":["Zhiyuan Zhao","Xueying Ding","B. Aditya Prakash"],"abstract":"Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs). However, conventional PINNs, relying on multilayer perceptrons (MLP), neglect the crucial temporal dependencies inherent in practical physics systems and thus fail to propagate the initial condition constraints globally and accurately capture the true solutions under various scenarios. In this paper, we introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer can accurately approximate PDE solutions by utilizing multi-head attention mechanisms to capture temporal dependencies. PINNsFormer transforms point-wise inputs into pseudo sequences and replaces point-wise PINNs loss with a sequential loss. Additionally, it incorporates a novel activation function, Wavelet, which anticipates Fourier decomposition through deep neural networks. Empirical results demonstrate that PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs. Moreover, PINNsFormer offers flexibility in integrating existing learning schemes for PINNs, further enhancing its performance.","url_abs":"https://arxiv.org/abs/2307.11833v3","url_pdf":"https://arxiv.org/pdf/2307.11833v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pinnsformer-a-transformer-based-framework-for","repo_url":"https://github.com/adityalab/pinnsformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.11833","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.11833"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adityalab/pinnsformer","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_honours":1},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"b3ba90157cf036b3","entry":"get_data","repo":"adityalab/pinnsformer","repo_kind":"official","path":"util.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3ba90157cf036b3"}},{"code_sha256_prefix":"608920c78f1ae5e9","entry":"get_n_params","repo":"adityalab/pinnsformer","repo_kind":"official","path":"util.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"608920c78f1ae5e9"}},{"code_sha256_prefix":"2530e16bed546c6a","entry":"group_add","repo":"adityalab/pinnsformer","repo_kind":"official","path":"pyhessian.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/pyhessian.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2530e16bed546c6a"}},{"code_sha256_prefix":"c35e05b48a3f5b73","entry":"group_product","repo":"adityalab/pinnsformer","repo_kind":"official","path":"pyhessian.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/pyhessian.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c35e05b48a3f5b73"}},{"code_sha256_prefix":"9b846f94a5604465","entry":"make_time_sequence","repo":"adityalab/pinnsformer","repo_kind":"official","path":"util.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b846f94a5604465"}},{"code_sha256_prefix":"0e4f4ed57a87f93d","entry":"normalization","repo":"adityalab/pinnsformer","repo_kind":"official","path":"pyhessian.py","file_url":"https://github.com/adityalab/pinnsformer/blob/HEAD/pyhessian.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0e4f4ed57a87f93d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}