FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions

• URL: http://arxiv.org/abs/2410.23405v1
• Date: Wed, 30 Oct 2024 19:15:43 GMT
• Title: FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions
• Authors: Anuroop Sriram, Benjamin Kurt Miller, Ricky T. Q. Chen, Brandon M. Wood,
• Abstract summary: FlowLLM is a novel generative model that combines large language models (LLMs) and Riemannian flow matching (RFM) to design novel crystalline materials. Our approach significantly outperforms state-of-the-art methods, increasing the generation rate of stable materials by over three times and increasing the rate for stable, unique, and novel crystals by $sim50%$.
• Score: 16.68310253042657
• License:
• Abstract: Material discovery is a critical area of research with the potential to revolutionize various fields, including carbon capture, renewable energy, and electronics. However, the immense scale of the chemical space makes it challenging to explore all possible materials experimentally. In this paper, we introduce FlowLLM, a novel generative model that combines large language models (LLMs) and Riemannian flow matching (RFM) to design novel crystalline materials. FlowLLM first fine-tunes an LLM to learn an effective base distribution of meta-stable crystals in a text representation. After converting to a graph representation, the RFM model takes samples from the LLM and iteratively refines the coordinates and lattice parameters. Our approach significantly outperforms state-of-the-art methods, increasing the generation rate of stable materials by over three times and increasing the rate for stable, unique, and novel crystals by $\sim50\%$ - a huge improvement on a difficult problem. Additionally, the crystals generated by FlowLLM are much closer to their relaxed state when compared with another leading model, significantly reducing post-hoc computational cost.

Related papers

• FlowMM: Generating Materials with Riemannian Flow Matching [16.68310253042657]
  We present FlowMM, a pair of generative models that achieve state-of-the-art performance on both tasks. Our framework enables the freedom to choose the flow base distributions, drastically simplifying the problem of learning crystal structures. In addition to standard benchmarks, we validate FlowMM's generated structures with quantum chemistry calculations.
  arXiv  Detail & Related papers  (2024-06-07T07:46:23Z)
• Multi-fidelity surrogate with heterogeneous input spaces for modeling melt pools in laser-directed energy deposition [0.0]
  Multi-fidelity (MF) modeling is a powerful statistical approach that can intelligently blend data from varied fidelity sources. One major challenge in using MF surrogates to merge a hierarchy of melt pool models is the variability in input spaces. This paper introduces a novel approach for constructing an MF surrogate for predicting melt pool geometry by integrating models of varying complexity.
  arXiv  Detail & Related papers  (2024-03-19T20:12:46Z)
• A Microstructure-based Graph Neural Network for Accelerating Multiscale Simulations [0.0]
  We introduce an alternative surrogate modeling strategy that allows for keeping the multiscale nature of the problem. We achieve this by predicting full-field microscopic strains using a graph neural network (GNN) while retaining the microscopic material model. We demonstrate for several challenging scenarios that the surrogate can predict complex macroscopic stress-strain paths.
  arXiv  Detail & Related papers  (2024-02-20T15:54:24Z)
• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text [57.01994216693825]
  Fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable. We show that our strongest model can generate materials predicted to be metastable at about twice the rate of CDVAE. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material.
  arXiv  Detail & Related papers  (2024-02-06T20:35:28Z)
• Scalable Diffusion for Materials Generation [99.71001883652211]
  We develop a unified crystal representation that can represent any crystal structure (UniMat) UniMat can generate high fidelity crystal structures from larger and more complex chemical systems. We propose additional metrics for evaluating generative models of materials.
  arXiv  Detail & Related papers  (2023-10-18T15:49:39Z)
• SE(3)-Stochastic Flow Matching for Protein Backbone Generation [54.951832422425454]
  We introduce FoldFlow, a series of novel generative models of increasing modeling power based on the flow-matching paradigm over $3mathrmD$ rigid motions. Our family of FoldFlowgenerative models offers several advantages over previous approaches to the generative modeling of proteins.
  arXiv  Detail & Related papers  (2023-10-03T19:24:24Z)
• Latent Diffusion Energy-Based Model for Interpretable Text Modeling [104.85356157724372]
  We introduce a novel symbiosis between the diffusion models and latent space EBMs in a variational learning framework. We develop a geometric clustering-based regularization jointly with the information bottleneck to further improve the quality of the learned latent space.
  arXiv  Detail & Related papers  (2022-06-13T03:41:31Z)
• Normalizing flows for atomic solids [67.70049117614325]
  We present a machine-learning approach, based on normalizing flows, for modelling atomic solids. We report Helmholtz free energy estimates for cubic and hexagonal ice modelled as monatomic water as well as for a truncated and shifted Lennard-Jones system. Our results thus demonstrate that normalizing flows can provide high-quality samples and free energy estimates of solids, without the need for multi-staging or for imposing restrictions on the crystal geometry.
  arXiv  Detail & Related papers  (2021-11-16T18:54:49Z)
• Moser Flow: Divergence-based Generative Modeling on Manifolds [49.04974733536027]
  Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF) MF does not require invoking or backpropagating through an ODE solver during training. We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
  arXiv  Detail & Related papers  (2021-08-18T09:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.