{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/forces-are-not-enough-benchmark-and-critical","title":"Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations","arxiv_id":"2210.07237","date":"2022-10-13","proceeding":null,"authors":["Xiang Fu","Zhenghao Wu","Wujie Wang","Tian Xie","Sinan Keten","Rafael Gomez-Bombarelli","Tommi Jaakkola"],"abstract":"Molecular dynamics (MD) simulation techniques are widely used for various natural science applications. Increasingly, machine learning (ML) force field (FF) models begin to replace ab-initio simulations by predicting forces directly from atomic structures. Despite significant progress in this area, such techniques are primarily benchmarked by their force/energy prediction errors, even though the practical use case would be to produce realistic MD trajectories. We aim to fill this gap by introducing a novel benchmark suite for learned MD simulation. We curate representative MD systems, including water, organic molecules, a peptide, and materials, and design evaluation metrics corresponding to the scientific objectives of respective systems. We benchmark a collection of state-of-the-art (SOTA) ML FF models and illustrate, in particular, how the commonly benchmarked force accuracy is not well aligned with relevant simulation metrics. We demonstrate when and how selected SOTA methods fail, along with offering directions for further improvement. Specifically, we identify stability as a key metric for ML models to improve. Our benchmark suite comes with a comprehensive open-source codebase for training and simulation with ML FFs to facilitate future work.","url_abs":"https://arxiv.org/abs/2210.07237v2","url_pdf":"https://arxiv.org/pdf/2210.07237v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"forces-are-not-enough-benchmark-and-critical","repo_url":"https://github.com/kyonofx/mdsim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"forces-are-not-enough-benchmark-and-critical","repo_url":"https://github.com/ask-berkeley/stable-training","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"forces-are-not-enough-benchmark-and-critical","repo_url":"https://github.com/nec-research/picps-ml4sci","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.07237","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.07237"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ask-berkeley/stable-training","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nec-research/picps-ml4sci","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kyonofx/mdsim","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2fbcf403f22839cf","entry":"energy_mae","repo":"kyonofx/mdsim","repo_kind":"official","path":"mdsim/modules/evaluator.py","file_url":"https://github.com/kyonofx/mdsim/blob/HEAD/mdsim/modules/evaluator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fbcf403f22839cf"}},{"code_sha256_prefix":"2c8a340cb31208e6","entry":"energy_mse","repo":"kyonofx/mdsim","repo_kind":"official","path":"mdsim/modules/evaluator.py","file_url":"https://github.com/kyonofx/mdsim/blob/HEAD/mdsim/modules/evaluator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c8a340cb31208e6"}},{"code_sha256_prefix":"73c0807b6656e859","entry":"energy_rmse","repo":"kyonofx/mdsim","repo_kind":"official","path":"mdsim/modules/evaluator.py","file_url":"https://github.com/kyonofx/mdsim/blob/HEAD/mdsim/modules/evaluator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"73c0807b6656e859"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}