Information fusion strategy integrating pre-trained language model and contrastive learning for materials knowledge mining

• URL: http://arxiv.org/abs/2506.12516v1
• Date: Sat, 14 Jun 2025 14:09:00 GMT
• Title: Information fusion strategy integrating pre-trained language model and contrastive learning for materials knowledge mining
• Authors: Yongqian Peng, Zhouran Zhang, Longhui Zhang, Fengyuan Zhao, Yahao Li, Yicong Ye, Shuxin Bai,
• Abstract summary: Machine learning has revolutionized materials design, yet predicting complex properties like alloy ductility remains challenging.<n>Here, we present an innovative information fusion architecture that integrates domain-specific texts from materials science literature with quantitative physical descriptors to overcome these limitations.<n>Our framework employs MatSciBERT for advanced textual comprehension and incorporates contrastive learning to automatically extract implicit knowledge regarding processing parameters and microstructural characteristics.
• Score: 0.4128284355136163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning has revolutionized materials design, yet predicting complex properties like alloy ductility remains challenging due to the influence of processing conditions and microstructural features that resist quantification through traditional reductionist approaches. Here, we present an innovative information fusion architecture that integrates domain-specific texts from materials science literature with quantitative physical descriptors to overcome these limitations. Our framework employs MatSciBERT for advanced textual comprehension and incorporates contrastive learning to automatically extract implicit knowledge regarding processing parameters and microstructural characteristics. Through rigorous ablation studies and comparative experiments, the model demonstrates superior performance, achieving coefficient of determination (R2) values of 0.849 and 0.680 on titanium alloy validation set and refractory multi-principal-element alloy test set. This systematic approach provides a holistic framework for property prediction in complex material systems where quantitative descriptors are incomplete and establishes a foundation for knowledge-guided materials design and informatics-driven materials discovery.

Related papers

• Towards Agentic Intelligence for Materials Science [73.4576385477731]
  This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining to goal-conditioned agents interfacing with simulation and experimental platforms.<n>To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science.
  arXiv  Detail & Related papers  (2026-01-29T23:48:43Z)
• Causal Discovery from Data Assisted by Large Language Models [50.193740129296245]
  It is essential to integrate experimental data with prior domain knowledge for knowledge driven discovery.<n>Here we demonstrate this approach by combining high-resolution scanning transmission electron microscopy (STEM) data with insights derived from large language models (LLMs)<n>By fine-tuning ChatGPT on domain-specific literature, we construct adjacency matrices for Directed Acyclic Graphs (DAGs) that map the causal relationships between structural, chemical, and polarization degrees of freedom in Sm-doped BiFeO3 (SmBFO)
  arXiv  Detail & Related papers  (2025-03-18T02:14:49Z)
• Universal Semantic Embeddings of Chemical Elements for Enhanced Materials Inference and Discovery [10.842037420887468]
  We present a framework for generating universal semantic embeddings of chemical elements to advance materials inference and discovery.<n>This framework leverages ElementBERT, a domain-specific BERT-based natural language processing model trained on 1.29 million abstracts of alloy-related scientific papers.
  arXiv  Detail & Related papers  (2025-02-19T07:26:03Z)
• DARWIN 1.5: Large Language Models as Materials Science Adapted Learners [46.7259033847682]
  We propose DARWIN 1.5, the largest open-source large language model tailored for materials science.<n> DARWIN eliminates the need for task-specific descriptors and enables a flexible, unified approach to material property prediction and discovery.<n>Our approach integrates 6M material domain papers and 21 experimental datasets from 49,256 materials across modalities while enabling cross-task knowledge transfer.
  arXiv  Detail & Related papers  (2024-12-16T16:51:27Z)
• Material Property Prediction with Element Attribute Knowledge Graphs and Multimodal Representation Learning [8.523289773617503]
  We build an element property knowledge graph and utilize an embedding model to encode the element attributes within the knowledge graph. A multimodal fusion framework, ESNet, integrates element property features with crystal structure features to generate joint multimodal representations. This provides a more comprehensive perspective for predicting the performance of crystalline materials.
  arXiv  Detail & Related papers  (2024-11-13T08:07:21Z)
• Foundation Model for Composite Microstructures: Reconstruction, Stiffness, and Nonlinear Behavior Prediction [0.0]
  We present a foundation model specifically designed for composite materials.<n>Our findings validate the feasibility and effectiveness of foundation models in composite materials.<n>This framework enables high-accuracy predictions even when experimental data are scarce.
  arXiv  Detail & Related papers  (2024-11-10T19:06:25Z)
• From Tokens to Materials: Leveraging Language Models for Scientific Discovery [12.211984932142537]
  This study investigates the application of language model embeddings to enhance material property prediction in materials science. We demonstrate that domain-specific models, particularly MatBERT, significantly outperform general-purpose models in extracting implicit knowledge from compound names and material properties.
  arXiv  Detail & Related papers  (2024-10-21T16:31:23Z)
• Compositional Representation of Polymorphic Crystalline Materials [56.80318252233511]
  We introduce PCRL, a novel approach that employs probabilistic modeling of composition to capture the diverse polymorphs from available structural information.<n>Extensive evaluations on sixteen datasets demonstrate the effectiveness of PCRL in learning compositional representation.
  arXiv  Detail & Related papers  (2023-11-17T20:34:28Z)
• Multimodal machine learning for materials science: composition-structure bimodal learning for experimentally measured properties [4.495968252019426]
  This paper introduces a novel approach to multimodal machine learning in materials science via composition-structure bimodal learning. The proposed COmposition-Structure Bimodal Network (COSNet) is designed to enhance learning and predictions of experimentally measured materials properties that have incomplete structure information.
  arXiv  Detail & Related papers  (2023-08-04T02:04:52Z)
• Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation [72.19108372163868]
  We propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation.<n>For structural texture knowledge, we introduce a Contourlet Decomposition Module (CDM) that decomposes low-level features.<n>For statistical texture knowledge, we propose a Denoised Texture Intensity Equalization Module (DTIEM) to adaptively extract and enhance statistical texture knowledge.
  arXiv  Detail & Related papers  (2023-05-06T06:01:11Z)
• Data-driven multi-scale modeling and robust optimization of composite structure with uncertainty quantification [0.42581756453559755]
  This chapter demonstrates advanced data-driven methods and outlines the capability that must be developed/added for the multi-scale modeling of advanced composite materials. It proposes a multi-scale modeling method for composite structures based on a finite element method (FEM) simulation driven by surrogate models/emulators.
  arXiv  Detail & Related papers  (2022-10-13T16:40:11Z)
• Meta-learning using privileged information for dynamics [66.32254395574994]
  We extend the Neural ODE Process model to use additional information within the Learning Using Privileged Information setting. We validate our extension with experiments showing improved accuracy and calibration on simulated dynamics tasks.
  arXiv  Detail & Related papers  (2021-04-29T12:18:02Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.