{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/comprehensive-molecular-representation-from","title":"Comprehensive Molecular Representation from Equivariant Transformer","arxiv_id":"2308.10752","date":"2023-08-21","proceeding":null,"authors":["Nianze Tao","Hiromi Morimoto","Stefano Leoni"],"abstract":"The tradeoff between precision and performance in molecular simulations can nowadays be addressed by machine-learned force fields (MLFF), which combine \\textit{ab initio} accuracy with force field numerical efficiency. Different from conventional force fields however, incorporating relevant electronic degrees of freedom into MLFFs becomes important. Here, we implement an equivariant transformer that embeds molecular net charge and spin state without additional neural network parameters. The model trained on a singlet/triplet non-correlated \\ce{CH2} dataset can identify different spin states and shows state-of-the-art extrapolation capability. Therein, self-attention sensibly captures non-local effects, which, as we show, can be finely tuned over the network hyper-parameters. We indeed found that Softmax activation functions utilised in the self-attention mechanism of graph networks outperformed ReLU-like functions in prediction accuracy. Increasing the attention temperature from $\\tau = \\sqrt{d}$ to $\\sqrt{2d}$ further improved the extrapolation capability, indicating a weighty role of nonlocality. Additionally, a weight initialisation method was purposed that sensibly accelerated the training process.","url_abs":"https://arxiv.org/abs/2308.10752v2","url_pdf":"https://arxiv.org/pdf/2308.10752v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"comprehensive-molecular-representation-from","repo_url":"https://github.com/augus1999/torch_cmret","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}