Regression with Large Language Models for Materials and Molecular Property Prediction

• URL: http://arxiv.org/abs/2409.06080v1
• Date: Mon, 9 Sep 2024 21:26:32 GMT
• Title: Regression with Large Language Models for Materials and Molecular Property Prediction
• Authors: Ryan Jacobs, Maciej P. Polak, Lane E. Schultz, Hamed Mahdavi, Vasant Honavar, Dane Morgan,
• Abstract summary: We demonstrate the ability of large language models (LLMs) to perform material and molecular property regression tasks. We benchmark the Large Language Model Meta AI (LLaMA) 3 on several molecular properties in the QM9 dataset and 24 materials properties.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We demonstrate the ability of large language models (LLMs) to perform material and molecular property regression tasks, a significant deviation from the conventional LLM use case. We benchmark the Large Language Model Meta AI (LLaMA) 3 on several molecular properties in the QM9 dataset and 24 materials properties. Only composition-based input strings are used as the model input and we fine tune on only the generative loss. We broadly find that LLaMA 3, when fine-tuned using the SMILES representation of molecules, provides useful regression results which can rival standard materials property prediction models like random forest or fully connected neural networks on the QM9 dataset. Not surprisingly, LLaMA 3 errors are 5-10x higher than those of the state-of-the-art models that were trained using far more granular representation of molecules (e.g., atom types and their coordinates) for the same task. Interestingly, LLaMA 3 provides improved predictions compared to GPT-3.5 and GPT-4o. This work highlights the versatility of LLMs, suggesting that LLM-like generative models can potentially transcend their traditional applications to tackle complex physical phenomena, thus paving the way for future research and applications in chemistry, materials science and other scientific domains.

Related papers

• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• An Empirical Study of LLaMA3 Quantization: From LLMs to MLLMs [54.91212829143966]
  This study explores LLaMA3's capabilities when quantized to low bit-width. We evaluate 10 existing post-training quantization and LoRA-finetuning methods of LLaMA3 on 1-8 bits and diverse datasets. Our experimental results indicate that LLaMA3 still suffers non-negligent degradation in linguistic and visual contexts.
  arXiv  Detail & Related papers  (2024-04-22T10:03:03Z)
• Benchmarking Large Language Models for Molecule Prediction Tasks [7.067145619709089]
  Large Language Models (LLMs) stand at the forefront of a number of Natural Language Processing (NLP) tasks. This paper explores a fundamental question: Can LLMs effectively handle molecule prediction tasks? We identify several classification and regression prediction tasks across six standard molecule datasets. We compare their performance with existing Machine Learning (ML) models, which include text-based models and those specifically designed for analysing the geometric structure of molecules.
  arXiv  Detail & Related papers  (2024-03-08T05:59:56Z)
• Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension [63.330262740414646]
  We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs) We suggest investigating internal activations and quantifying LLM's truthfulness using the local intrinsic dimension (LID) of model activations.
  arXiv  Detail & Related papers  (2024-02-28T04:56:21Z)
• LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation [0.0]
  Large Language Models (LLMs) inherently lack long-term memory, making it a nontrivial, ad hoc, and inevitably biased task to fine-tune them on domain-specific literature and data. Here we introduce LLaMP, a framework of hierarchical reasoning-and-acting (RAG) agents that can interact with computational and experimental data. Without fine-tuning, LLaMP demonstrates strong tool usage ability to comprehend and integrate various modalities of materials science concepts.
  arXiv  Detail & Related papers  (2024-01-30T18:37:45Z)
• Can Large Language Models Understand Molecules? [0.0699049312989311]
  We investigate the performance of GPT and LLaMA compared to pre-trained models on SMILES in embedding SMILES strings on downstream tasks. We find that SMILES embeddings generated using LLaMA outperform those from GPT in both molecular property and DDI prediction tasks.
  arXiv  Detail & Related papers  (2024-01-05T18:31:34Z)
• Can Large Language Models Empower Molecular Property Prediction? [16.5246941211725]
  Molecular property prediction has gained significant attention due to its transformative potential in scientific disciplines. Recently, the rapid development of Large Language Models (LLMs) has revolutionized the field of NLP. In this work, we advance towards this objective through two perspectives: zero/few-shot molecular classification, and using the new explanations generated by LLMs as representations of molecules.
  arXiv  Detail & Related papers  (2023-07-14T16:06:42Z)
• Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective [53.300288393173204]
  Large Language Models (LLMs) have shown remarkable performance in various cross-modal tasks. In this work, we propose an In-context Few-Shot Molecule Learning paradigm for molecule-caption translation. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation.
  arXiv  Detail & Related papers  (2023-06-11T08:16:25Z)
• Prediction of transport property via machine learning molecular movements [1.0554048699217666]
  We present a simple supervised machine learning method to predict the transport properties of materials. This method was applied to predict the viscosity of lubricant molecules in confinement with shear flow. We revealed two types of molecular mechanisms that contribute to low viscosity.
  arXiv  Detail & Related papers  (2022-03-07T02:28:07Z)
• Prediction of liquid fuel properties using machine learning models with Gaussian processes and probabilistic conditional generative learning [56.67751936864119]
  The present work aims to construct cheap-to-compute machine learning (ML) models to act as closure equations for predicting the physical properties of alternative fuels. Those models can be trained using the database from MD simulations and/or experimental measurements in a data-fusion-fidelity approach. The results show that ML models can predict accurately the fuel properties of a wide range of pressure and temperature conditions.
  arXiv  Detail & Related papers  (2021-10-18T14:43:50Z)
• BIGDML: Towards Exact Machine Learning Force Fields for Materials [55.944221055171276]
  Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning approach and demonstrate its ability to construct reliable force fields using a training set with just 10-200 atoms.
  arXiv  Detail & Related papers  (2021-06-08T10:14:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.