Large Language Models Are Innate Crystal Structure Generators

• URL: http://arxiv.org/abs/2502.20933v1
• Date: Fri, 28 Feb 2025 10:41:16 GMT
• Title: Large Language Models Are Innate Crystal Structure Generators
• Authors: Jingru Gan, Peichen Zhong, Yuanqi Du, Yanqiao Zhu, Chenru Duan, Haorui Wang, Carla P. Gomes, Kristin A. Persson, Daniel Schwalbe-Koda, Wei Wang,
• Abstract summary: We show that pre-trained Large Language Models can inherently generate stable crystal structures without additional training.<n>Our framework MatLLMSearch integrates pre-trained LLMs with evolutionary search algorithms, achieving a 78.38% metastable rate.
• Score: 30.44669215588058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Crystal structure generation is fundamental to materials discovery, enabling the prediction of novel materials with desired properties. While existing approaches leverage Large Language Models (LLMs) through extensive fine-tuning on materials databases, we show that pre-trained LLMs can inherently generate stable crystal structures without additional training. Our novel framework MatLLMSearch integrates pre-trained LLMs with evolutionary search algorithms, achieving a 78.38% metastable rate validated by machine learning interatomic potentials and 31.7% DFT-verified stability via quantum mechanical calculations, outperforming specialized models such as CrystalTextLLM. Beyond crystal structure generation, we further demonstrate that our framework can be readily adapted to diverse materials design tasks, including crystal structure prediction and multi-objective optimization of properties such as deformation energy and bulk modulus, all without fine-tuning. These results establish pre-trained LLMs as versatile and effective tools for materials discovery, opening up new venues for crystal structure generation with reduced computational overhead and broader accessibility.

Related papers

• CrystalFormer-RL: Reinforcement Fine-Tuning for Materials Design [2.290956583394892]
  We explore the applications of reinforcement fine-tuning to the autoregressive transformer-based materials generative model CrystalFormer. By optimizing reward signals, fine-tuning infuses knowledge from discriminative models into generative models. The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties.
  arXiv  Detail & Related papers  (2025-04-03T07:59:30Z)
• Transformer-Enhanced Variational Autoencoder for Crystal Structure Prediction [16.846289696048647]
  We propose a Transformer-Enhanced Variational Autoencoder for Crystal Structure Prediction (TransVAE-CSP)<n>TransVAE-CSP learns the characteristic distribution space of stable materials, enabling both the reconstruction and generation of crystal structures.<n> Experimental results on the carbon_24, perov_5, and mp_20 datasets demonstrate that TransVAE-CSP outperforms existing methods in structure reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2025-02-13T15:45:36Z)
• Symmetry-Aware Bayesian Flow Networks for Crystal Generation [0.562479170374811]
  We introduce SymmBFN, a novel symmetry-aware Bayesian Flow Network (BFN) for crystalline material generation.<n>SymmBFN substantially improves efficiency, generating stable structures at least 50 times faster than the next-best method.<n>Our findings establish BFNs as an effective tool for accelerating the discovery of crystalline materials.
  arXiv  Detail & Related papers  (2025-02-05T13:14:50Z)
• Efficient Symmetry-Aware Materials Generation via Hierarchical Generative Flow Networks [52.13486402193811]
  New solid-state materials require rapidly exploring the vast space of crystal structures and locating stable regions. Existing methods struggle to explore large material spaces and generate diverse samples with desired properties and requirements. We propose a novel generative model employing a hierarchical exploration strategy to efficiently exploit the symmetry of the materials space to generate crystal structures given desired properties.
  arXiv  Detail & Related papers  (2024-11-06T23:53:34Z)
• Generative Hierarchical Materials Search [91.93125016916463]
  We propose Generative Hierarchical Materials Search (GenMS) for controllable generation of crystal structures. GenMS consists of (1) a language model that takes high-level natural language as input and generates intermediate textual information about a crystal. GenMS additionally uses a graph neural network to predict properties (e.g., formation energy) from the generated crystal structures.
  arXiv  Detail & Related papers  (2024-09-10T17:51:28Z)
• Universal Machine Learning Kohn-Sham Hamiltonian for Materials [5.189794091596078]
  This study introduces a universal electronic Hamiltonian model trained on Hamiltonian matrices. We demonstrate its generality in predicting electronic structures across the whole periodic table. By offering a reliable framework for computing electronic properties, this universal Hamiltonian model lays the groundwork for advancements in diverse fields.
  arXiv  Detail & Related papers  (2024-02-14T15:38:56Z)
• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text [57.01994216693825]
  Fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable. We show that our strongest model can generate materials predicted to be metastable at about twice the rate of CDVAE. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material.
  arXiv  Detail & Related papers  (2024-02-06T20:35:28Z)
• Scalable Diffusion for Materials Generation [99.71001883652211]
  We develop a unified crystal representation that can represent any crystal structure (UniMat) UniMat can generate high fidelity crystal structures from larger and more complex chemical systems. We propose additional metrics for evaluating generative models of materials.
  arXiv  Detail & Related papers  (2023-10-18T15:49:39Z)
• Latent Conservative Objective Models for Data-Driven Crystal Structure Prediction [62.36797874900395]
  In computational chemistry, crystal structure prediction is an optimization problem. One approach to tackle this problem involves building simulators based on density functional theory (DFT) followed by running search in simulation. We show that our approach, dubbed LCOMs (latent conservative objective models), performs comparably to the best current approaches in terms of success rate of structure prediction.
  arXiv  Detail & Related papers  (2023-10-16T04:35:44Z)
• Crystal-GFN: sampling crystals with desirable properties and constraints [103.79058968784163]
  We introduce Crystal-GFN, a generative model of crystal structures that sequentially samples structural properties of crystalline materials. In this paper, we use as objective the formation energy per atom of a crystal structure predicted by a new proxy machine learning model trained on MatBench. The results demonstrate that Crystal-GFN is able to sample highly diverse crystals with low (median -3.1 eV/atom) predicted formation energy.
  arXiv  Detail & Related papers  (2023-10-07T21:36:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.