MechGPT, a language-based strategy for mechanics and materials modeling
that connects knowledge across scales, disciplines and modalities
- URL: http://arxiv.org/abs/2310.10445v1
- Date: Mon, 16 Oct 2023 14:29:35 GMT
- Title: MechGPT, a language-based strategy for mechanics and materials modeling
that connects knowledge across scales, disciplines and modalities
- Authors: Markus J. Buehler
- Abstract summary: We use a Large Language Model (LLM) to distill question-answer pairs from raw sources followed by fine-tuning.
The resulting MechGPT LLM foundation model is used in a series of computational experiments to explore its capacity for knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across disparate areas.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: For centuries, researchers have sought out ways to connect disparate areas of
knowledge. While early scholars (Galileo, da Vinci, etc.) were experts across
fields, specialization has taken hold later. With the advent of Artificial
Intelligence, we can now explore relationships across areas (e.g.,
mechanics-biology) or disparate domains (e.g., failure mechanics-art). To
achieve this, we use a fine-tuned Large Language Model (LLM), here for a subset
of knowledge in multiscale materials failure. The approach includes the use of
a general-purpose LLM to distill question-answer pairs from raw sources
followed by LLM fine-tuning. The resulting MechGPT LLM foundation model is used
in a series of computational experiments to explore its capacity for knowledge
retrieval, various language tasks, hypothesis generation, and connecting
knowledge across disparate areas. While the model has some ability to recall
knowledge from training, we find that LLMs are particularly useful to extract
structural insights through Ontological Knowledge Graphs. These interpretable
graph structures provide explanatory insights, frameworks for new research
questions, and visual representations of knowledge that also can be used in
retrieval-augmented generation. Three versions of MechGPT are discussed,
featuring different sizes from 13 billion to 70 billion parameters, and
reaching context lengths of more than 10,000 tokens. This provides ample
capacity for sophisticated retrieval augmented strategies, as well as
agent-based modeling where multiple LLMs interact collaboratively and/or
adversarially, the incorporation of new data from the literature or web
searches, as well as multimodality.
Related papers
- Adapter-based Approaches to Knowledge-enhanced Language Models -- A Survey [48.52320309766703]
Knowledge-enhanced language models (KELMs) have emerged as promising tools to bridge the gap between large-scale language models and domain-specific knowledge.
KELMs can achieve higher factual accuracy and hallucinations by leveraging knowledge graphs (KGs)
arXiv Detail & Related papers (2024-11-25T14:10:24Z) - GIVE: Structured Reasoning with Knowledge Graph Inspired Veracity Extrapolation [108.2008975785364]
Graph Inspired Veracity Extrapolation (GIVE) is a novel reasoning framework that integrates the parametric and non-parametric memories.
Our method facilitates a more logical and step-wise reasoning approach akin to experts' problem-solving, rather than gold answer retrieval.
arXiv Detail & Related papers (2024-10-11T03:05:06Z) - Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-based Causal Discovery [10.573861741540853]
KG Structure as Prompt is a novel approach for integrating structural information from a knowledge graph, such as common neighbor nodes and metapaths, into prompt-based learning.
Experimental results on three types of biomedical and open-domain datasets under few-shot settings demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2024-07-26T14:07:00Z) - Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML)
This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature.
The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
arXiv Detail & Related papers (2024-07-17T20:01:21Z) - A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation.
Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge.
RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
arXiv Detail & Related papers (2024-05-10T02:48:45Z) - Materials science in the era of large language models: a perspective [0.0]
Large Language Models (LLMs) have garnered considerable interest due to their impressive capabilities.
This paper argues their ability to handle ambiguous requirements across a range of tasks and disciplines mean they could be a powerful tool to aid researchers.
arXiv Detail & Related papers (2024-03-11T17:34:25Z) - LLM Inference Unveiled: Survey and Roofline Model Insights [62.92811060490876]
Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges.
Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model.
This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems.
arXiv Detail & Related papers (2024-02-26T07:33:05Z) - Quantitative knowledge retrieval from large language models [4.155711233354597]
Large language models (LLMs) have been extensively studied for their abilities to generate convincing natural language sequences.
This paper explores the feasibility of LLMs as a mechanism for quantitative knowledge retrieval to aid data analysis tasks.
arXiv Detail & Related papers (2024-02-12T16:32:37Z) - Generative Multi-Modal Knowledge Retrieval with Large Language Models [75.70313858231833]
We propose an innovative end-to-end generative framework for multi-modal knowledge retrieval.
Our framework takes advantage of the fact that large language models (LLMs) can effectively serve as virtual knowledge bases.
We demonstrate significant improvements ranging from 3.0% to 14.6% across all evaluation metrics when compared to strong baselines.
arXiv Detail & Related papers (2024-01-16T08:44:29Z) - Generative retrieval-augmented ontologic graph and multi-agent
strategies for interpretive large language model-based materials design [0.0]
Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design and manufacturing.
Here we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials.
arXiv Detail & Related papers (2023-10-30T20:31:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.