DARWIN 1.5: Large Language Models as Materials Science Adapted Learners

• URL: http://arxiv.org/abs/2412.11970v2
• Date: Thu, 23 Jan 2025 08:07:41 GMT
• Title: DARWIN 1.5: Large Language Models as Materials Science Adapted Learners
• Authors: Tong Xie, Yuwei Wan, Yixuan Liu, Yuchen Zeng, Shaozhou Wang, Wenjie Zhang, Clara Grazian, Chunyu Kit, Wanli Ouyang, Dongzhan Zhou, Bram Hoex,
• Abstract summary: We propose DARWIN 1.5, the largest open-source large language model tailored for materials science.<n> DARWIN eliminates the need for task-specific descriptors and enables a flexible, unified approach to material property prediction and discovery.<n>Our approach integrates 6M material domain papers and 21 experimental datasets from 49,256 materials across modalities while enabling cross-task knowledge transfer.
• Score: 46.7259033847682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Materials discovery and design aim to find compositions and structures with desirable properties over highly complex and diverse physical spaces. Traditional solutions, such as high-throughput simulations or machine learning, often rely on complex descriptors, which hinder generalizability and transferability across different material systems. Moreover, These descriptors may inadequately represent macro-scale material properties, which are influenced by structural imperfections and compositional variations in real-world samples, thus limiting their practical applicability. To address these challenges, we propose DARWIN 1.5, the largest open-source large language model tailored for materials science. By leveraging natural language as input, DARWIN eliminates the need for task-specific descriptors and enables a flexible, unified approach to material property prediction and discovery. Our approach integrates 6M material domain papers and 21 experimental datasets from 49,256 materials across modalities while enabling cross-task knowledge transfer. The enhanced model achieves up to 59.1% improvement in prediction accuracy over the base LLaMA-7B architecture and outperforms SOTA machine learning approaches across 8 materials design tasks. These results establish LLMs as a promising foundation for developing versatile and scalable models in materials science.

Related papers

• Inverse Materials Design by Large Language Model-Assisted Generative Framework [35.04390544440238]
  AlloyGAN is a framework that integrates Large Language Model (LLM)-assisted text mining with Conditional Generative Adversarial Networks (CGANs) For metallic glasses, the framework predicts thermodynamic properties with discrepancies of less than 8% from experiments. By bridging generative AI with domain knowledge, AlloyGAN offers a scalable approach to accelerate the discovery of materials with tailored properties.
  arXiv  Detail & Related papers  (2025-02-25T11:52:59Z)
• Foundational Large Language Models for Materials Research [22.77591279242839]
  Large Language Models (LLMs) offer opportunities to accelerate materials research through automated analysis and prediction. Here, we present LLaMat, a family of foundational models for materials science developed through continued pretraining of LLaMA models. We demonstrate that LLaMat excels in materials-specific NLP and structured information extraction while maintaining general linguistic capabilities.
  arXiv  Detail & Related papers  (2024-12-12T18:46:38Z)
• MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale [66.73529246309033]
  multimodal large language models (MLLMs) have shown significant potential in a broad range of multimodal tasks. Existing instruction-tuning datasets only provide phrase-level answers without any intermediate rationales. We introduce a scalable and cost-effective method to construct a large-scale multimodal instruction-tuning dataset with rich intermediate rationales.
  arXiv  Detail & Related papers  (2024-12-06T18:14:24Z)
• Foundation Model for Composite Microstructures: Reconstruction, Stiffness, and Nonlinear Behavior Prediction [0.0]
  We present a foundation model specifically designed for composite materials. Our findings validate the feasibility and effectiveness of foundation models in composite materials. This framework enables high-accuracy predictions even when experimental data are scarce.
  arXiv  Detail & Related papers  (2024-11-10T19:06:25Z)
• When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge? [15.318301783084681]
  Large language models (LLMs) can inadvertently learn and retain sensitive information and harmful content during training. We propose a lightweight unlearning framework based on Retrieval-Augmented Generation (RAG) technology. We evaluate our framework through extensive experiments on both open-source and closed-source models, including ChatGPT, Gemini, Llama-2-7b-chat-hf, and PaLM 2.
  arXiv  Detail & Related papers  (2024-10-20T03:51:01Z)
• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence [0.0]
  The proposed physics-aware generative AI platform, AtomAgents, synergizes the intelligence of large language models (LLM) Our results enable accurate prediction of key characteristics across alloys and highlight the crucial role of solid solution alloying to steer the development of advanced metallic alloys.
  arXiv  Detail & Related papers  (2024-07-13T22:46:02Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  This dataset includes figures such as schematic diagrams, simulated images, macroscopic/microscopic photos, and experimental visualizations. We developed benchmarks for scientific figure captioning and multiple-choice questions, evaluating six proprietary and over ten open-source models. The dataset and benchmarks will be released to support further research.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• OpenMaterial: A Comprehensive Dataset of Complex Materials for 3D Reconstruction [54.706361479680055]
  We introduce the OpenMaterial dataset, comprising 1001 objects made of 295 distinct materials. OpenMaterial provides comprehensive annotations, including 3D shape, material type, camera pose, depth, and object mask. It stands as the first large-scale dataset enabling quantitative evaluations of existing algorithms on objects with diverse and challenging materials.
  arXiv  Detail & Related papers  (2024-06-13T07:46:17Z)
• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text [57.01994216693825]
  Fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable. We show that our strongest model can generate materials predicted to be metastable at about twice the rate of CDVAE. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material.
  arXiv  Detail & Related papers  (2024-02-06T20:35:28Z)
• MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities [0.0]
  We use a Large Language Model (LLM) to distill question-answer pairs from raw sources followed by fine-tuning. The resulting MechGPT LLM foundation model is used in a series of computational experiments to explore its capacity for knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across disparate areas.
  arXiv  Detail & Related papers  (2023-10-16T14:29:35Z)
• Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks [90.11273439036455]
  Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks. We propose Knowledge-Augmented Reasoning Distillation (KARD), a novel method that fine-tunes small LMs to generate rationales from LLMs with augmented knowledge retrieved from an external knowledge base. We empirically show that KARD significantly improves the performance of small T5 and GPT models on the challenging knowledge-intensive reasoning datasets.
  arXiv  Detail & Related papers  (2023-05-28T13:00:00Z)
• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
  Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability. We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization. We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
  arXiv  Detail & Related papers  (2023-05-23T17:51:52Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
  We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations. We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
  arXiv  Detail & Related papers  (2023-04-28T21:48:31Z)
• A Comprehensive and Versatile Multimodal Deep Learning Approach for Predicting Diverse Properties of Advanced Materials [0.9517427900627922]
  We present a multimodal deep learning framework for predicting physical properties of a 10-dimensional acrylic polymer composite material. Our approach handles an 18-dimensional complexity, with 10 compositional inputs and 8 property outputs, successfully predicting 913,680 property data points across 114,210 composition conditions. This study advances future research on different materials and the development of more sophisticated models, drawing us closer to the ultimate goal of predicting all properties of all materials.
  arXiv  Detail & Related papers  (2023-03-29T02:42:17Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Data-driven multi-scale modeling and robust optimization of composite structure with uncertainty quantification [0.42581756453559755]
  This chapter demonstrates advanced data-driven methods and outlines the capability that must be developed/added for the multi-scale modeling of advanced composite materials. It proposes a multi-scale modeling method for composite structures based on a finite element method (FEM) simulation driven by surrogate models/emulators.
  arXiv  Detail & Related papers  (2022-10-13T16:40:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.