{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-self-attention-ansatz-for-ab-initio-quantum","title":"A Self-Attention Ansatz for Ab-initio Quantum Chemistry","arxiv_id":"2211.13672","date":"2022-11-24","proceeding":null,"authors":["Ingrid von Glehn","James S. Spencer","David Pfau"],"abstract":"We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schr\\\"odinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.","url_abs":"https://arxiv.org/abs/2211.13672v2","url_pdf":"https://arxiv.org/pdf/2211.13672v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-self-attention-ansatz-for-ab-initio-quantum","repo_url":"https://github.com/deepmind/ferminet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"a-self-attention-ansatz-for-ab-initio-quantum","repo_url":"https://github.com/deepqmc/deepqmc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-self-attention-ansatz-for-ab-initio-quantum","repo_url":"https://github.com/google-deepmind/ferminet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2211.13672","atlas_url":"https://app.syntology.ai/?focus=2211.13672","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.13672"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepqmc/deepqmc","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/ferminet","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/ferminet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"listed":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"66e0f2656c8741a8","entry":"get_shell","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/hamil.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/hamil.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66e0f2656c8741a8"}},{"code_sha256_prefix":"b3f1e108018a5cd1","entry":"init_ewm","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/ewm.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/ewm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3f1e108018a5cd1"}},{"code_sha256_prefix":"86306d8b4d11e1cc","entry":"init_multi_mol_multi_state_ewm","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/ewm.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/ewm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86306d8b4d11e1cc"}},{"code_sha256_prefix":"0f3c94d1c894b32b","entry":"rng_wrapper","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/observable.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/observable.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f3c94d1c894b32b"}},{"code_sha256_prefix":"501327c99ddffb76","entry":"ssp","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/hkext.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/hkext.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"501327c99ddffb76"}},{"code_sha256_prefix":"66d22be8c73c2a40","entry":"compute_mean_and_std","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/observable.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/observable.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66d22be8c73c2a40"}},{"code_sha256_prefix":"70d3f6e1d9fdf446","entry":"sparse_attention","repo":"deepqmc/deepqmc","repo_kind":"listed","path":"src/deepqmc/folxext.py","file_url":"https://github.com/deepqmc/deepqmc/blob/HEAD/src/deepqmc/folxext.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70d3f6e1d9fdf446"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}