CrystalFormer-RL: Reinforcement Fine-Tuning for Materials Design

• URL: http://arxiv.org/abs/2504.02367v1
• Date: Thu, 03 Apr 2025 07:59:30 GMT
• Title: CrystalFormer-RL: Reinforcement Fine-Tuning for Materials Design
• Authors: Zhendong Cao, Lei Wang,
• Abstract summary: We explore the applications of reinforcement fine-tuning to the autoregressive transformer-based materials generative model CrystalFormer.<n>By optimizing reward signals, fine-tuning infuses knowledge from discriminative models into generative models.<n>The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties.
• Score: 2.290956583394892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement fine-tuning has instrumental enhanced the instruction-following and reasoning abilities of large language models. In this work, we explore the applications of reinforcement fine-tuning to the autoregressive transformer-based materials generative model CrystalFormer (arXiv:2403.15734) using discriminative machine learning models such as interatomic potentials and property prediction models. By optimizing reward signals-such as energy above the convex hull and material property figures of merit-reinforcement fine-tuning infuses knowledge from discriminative models into generative models. The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties, such as substantial dielectric constant and band gap simultaneously. Notably, we observe that reinforcement fine-tuning enables not only the property-guided novel material design ability of generative pre-trained model but also unlocks property-driven material retrieval from the unsupervised pre-training dataset. Leveraging rewards from discriminative models to fine-tune materials generative models opens an exciting gateway to the synergies of the machine learning ecosystem for materials.

Related papers

• UniGenX: Unified Generation of Sequence and Structure with Autoregressive Diffusion [61.690978792873196]
  Existing approaches rely on either autoregressive sequence models or diffusion models. We propose UniGenX, a unified framework that combines autoregressive next-token prediction with conditional diffusion models. We validate the effectiveness of UniGenX on material and small molecule generation tasks.
  arXiv  Detail & Related papers  (2025-03-09T16:43:07Z)
• Large Language Models Are Innate Crystal Structure Generators [30.44669215588058]
  We show that pre-trained Large Language Models can inherently generate stable crystal structures without additional training.<n>Our framework MatLLMSearch integrates pre-trained LLMs with evolutionary search algorithms, achieving a 78.38% metastable rate.
  arXiv  Detail & Related papers  (2025-02-28T10:41:16Z)
• Inverse Materials Design by Large Language Model-Assisted Generative Framework [35.04390544440238]
  AlloyGAN is a framework that integrates Large Language Model (LLM)-assisted text mining with Conditional Generative Adversarial Networks (CGANs) For metallic glasses, the framework predicts thermodynamic properties with discrepancies of less than 8% from experiments. By bridging generative AI with domain knowledge, AlloyGAN offers a scalable approach to accelerate the discovery of materials with tailored properties.
  arXiv  Detail & Related papers  (2025-02-25T11:52:59Z)
• Active Learning for Conditional Inverse Design with Crystal Generation and Foundation Atomic Models [2.8408587358426725]
  We present an active learning framework that combines crystal generation models and foundation atomic models to enhance the accuracy and efficiency of inverse design.<n>Our framework is general to accommodate different crystal generation and foundation atomic models, and establishes a scalable approach for AI-driven materials discovery.
  arXiv  Detail & Related papers  (2025-02-24T09:15:38Z)
• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text [57.01994216693825]
  Fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable. We show that our strongest model can generate materials predicted to be metastable at about twice the rate of CDVAE. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material.
  arXiv  Detail & Related papers  (2024-02-06T20:35:28Z)
• Scalable Diffusion for Materials Generation [99.71001883652211]
  We develop a unified crystal representation that can represent any crystal structure (UniMat) UniMat can generate high fidelity crystal structures from larger and more complex chemical systems. We propose additional metrics for evaluating generative models of materials.
  arXiv  Detail & Related papers  (2023-10-18T15:49:39Z)
• On the Stability of Iterative Retraining of Generative Models on their own Data [56.153542044045224]
  We study the impact of training generative models on mixed datasets. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough. We empirically validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-09-30T16:41:04Z)
• Evaluating the diversity and utility of materials proposed by generative models [38.85523285991743]
  We show how one state-of-the-art generative model, the physics-guided crystal generation model, can be used as part of the inverse design process. Our findings suggest how generative models might be improved to enable better inverse design.
  arXiv  Detail & Related papers  (2023-08-09T14:42:08Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• How to See Hidden Patterns in Metamaterials with Interpretable Machine Learning [82.67551367327634]
  We develop a new interpretable, multi-resolution machine learning framework for finding patterns in the unit-cells of materials. Specifically, we propose two new interpretable representations of metamaterials, called shape-frequency features and unit-cell templates.
  arXiv  Detail & Related papers  (2021-11-10T21:19:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.