{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/crystalformer-rl-reinforcement-fine-tuning","title":"CrystalFormer-RL: Reinforcement Fine-Tuning for Materials Design","arxiv_id":"2504.02367","date":"2025-04-03","proceeding":null,"authors":["Zhendong Cao","Lei Wang"],"abstract":"Reinforcement fine-tuning has instrumental enhanced the instruction-following and reasoning abilities of large language models. In this work, we explore the applications of reinforcement fine-tuning to the autoregressive transformer-based materials generative model CrystalFormer (arXiv:2403.15734) using discriminative machine learning models such as interatomic potentials and property prediction models. By optimizing reward signals-such as energy above the convex hull and material property figures of merit-reinforcement fine-tuning infuses knowledge from discriminative models into generative models. The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties, such as substantial dielectric constant and band gap simultaneously. Notably, we observe that reinforcement fine-tuning enables not only the property-guided novel material design ability of generative pre-trained model but also unlocks property-driven material retrieval from the unsupervised pre-training dataset. Leveraging rewards from discriminative models to fine-tune materials generative models opens an exciting gateway to the synergies of the machine learning ecosystem for materials.","url_abs":"https://arxiv.org/abs/2504.02367v1","url_pdf":"https://arxiv.org/pdf/2504.02367v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"crystalformer-rl-reinforcement-fine-tuning","repo_url":"https://github.com/deepmodeling/crystalformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"band-gap","task_name":"Band Gap"},{"task_slug":"dielectric-constant","task_name":"Dielectric Constant"},{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"property-prediction","task_name":"Property Prediction"},{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"},{"task_slug":"multimodal-generation","task_name":"multimodal generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2504.02367","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.02367"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmodeling/crystalformer","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"b0661134b5b6db88","entry":"get_labels","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b0661134b5b6db88"}},{"code_sha256_prefix":"35f0761aa7c2e90f","entry":"make_classifier","repo":"deepmodeling/crystalformer","repo_kind":"official","path":"crystalformer/extension/model.py","file_url":"https://github.com/deepmodeling/crystalformer/blob/HEAD/crystalformer/extension/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35f0761aa7c2e90f"}},{"code_sha256_prefix":"52927c8d9d88b5d5","entry":"GLXYZAW_from_sample","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"52927c8d9d88b5d5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}