Layer-to-Layer Knowledge Mixing in Graph Neural Network for Chemical Property Prediction

• URL: http://arxiv.org/abs/2510.20236v1
• Date: Thu, 23 Oct 2025 05:42:31 GMT
• Title: Layer-to-Layer Knowledge Mixing in Graph Neural Network for Chemical Property Prediction
• Authors: Teng Jiek See, Daokun Zhang, Mario Boley, David K. Chalmers,
• Abstract summary: Layer-to-Layer Knowledge Mixing (LKM) is a self-knowledge distillation method that increases the accuracy of state-of-the-art GNNs.<n>LKM efficiently aggregates multi-hop and multi-scale information, enabling improved representation of both local and global molecular features.
• Score: 2.919475219863926
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Graph Neural Networks (GNNs) are the currently most effective methods for predicting molecular properties but there remains a need for more accurate models. GNN accuracy can be improved by increasing the model complexity but this also increases the computational cost and memory requirement during training and inference. In this study, we develop Layer-to-Layer Knowledge Mixing (LKM), a novel self-knowledge distillation method that increases the accuracy of state-of-the-art GNNs while adding negligible computational complexity during training and inference. By minimizing the mean absolute distance between pre-existing hidden embeddings of GNN layers, LKM efficiently aggregates multi-hop and multi-scale information, enabling improved representation of both local and global molecular features. We evaluated LKM using three diverse GNN architectures (DimeNet++, MXMNet, and PAMNet) using datasets of quantum chemical properties (QM9, MD17 and Chignolin). We found that the LKM method effectively reduces the mean absolute error of quantum chemical and biophysical property predictions by up to 9.8% (QM9), 45.3% (MD17 Energy), and 22.9% (Chignolin). This work demonstrates the potential of LKM to significantly improve the accuracy of GNNs for chemical property prediction without any substantial increase in training and inference cost.

Related papers

• Towards Improved Quantum Machine Learning for Molecular Force Fields [0.0]
  equivariant quantum neural networks (QNN) for generating molecular force fields, focusing on the rMD17 dataset.<n>We consider a QNN architecture based on previous research and point out shortcomings in the parametrization of the atomic environments.<n>We propose a revised QNN architecture that addresses these shortcomings.
  arXiv  Detail & Related papers  (2025-05-06T06:02:12Z)
• Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference [53.38082028252104]
  We introduce HG2M and HG2M+ to combine both HGNN's superior performance and relational's efficient inference.<n> HG2M directly trains students with node features as input and soft labels from teacher HGNNs as targets.<n> HG2Ms demonstrate a 379.24$times$ speedup in inference over HGNNs on the large-scale IGB-3M-19 dataset.
  arXiv  Detail & Related papers  (2024-11-21T11:39:09Z)
• Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning [52.28945097811129]
  We propose an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences.<n>We reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear.<n>Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources.
  arXiv  Detail & Related papers  (2024-10-08T14:27:28Z)
• Dumpling GNN: Hybrid GNN Enables Better ADC Payload Activity Prediction Based on Chemical Structure [53.76752789814785]
  DumplingGNN is a hybrid Graph Neural Network architecture specifically designed for predicting ADC payload activity based on chemical structure. We evaluate it on a comprehensive ADC payload dataset focusing on DNA Topoisomerase I inhibitors. It demonstrates exceptional accuracy (91.48%), sensitivity (95.08%), and specificity (97.54%) on our specialized ADC payload dataset.
  arXiv  Detail & Related papers  (2024-09-23T17:11:04Z)
• Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning [0.0]
  We introduce a Multi-Modal Fusion (MMF) framework that harnesses the analytical prowess of Graph Neural Networks (GNNs) and the linguistic generative and predictive abilities of Large Language Models (LLMs) Our framework combines the effectiveness of GNNs in modeling graph-structured data with the zero-shot and few-shot learning capabilities of LLMs, enabling improved predictions while reducing the risk of overfitting.
  arXiv  Detail & Related papers  (2024-08-27T11:10:39Z)
• Accelerating Molecular Graph Neural Networks via Knowledge Distillation [1.9116784879310031]
  Recent advances in graph neural networks (GNNs) have enabled more comprehensive modeling of molecules and molecular systems. As the field has been progressing to bigger and more complex architectures, state-of-the-art GNNs have become largely prohibitive for many large-scale applications. We devise KD strategies that facilitate the distillation of hidden representations in directional and equivariant GNNs, and evaluate their performance on the regression task of energy and force prediction.
  arXiv  Detail & Related papers  (2023-06-26T16:24:31Z)
• Multi-Task Mixture Density Graph Neural Networks for Predicting Cu-based Single-Atom Alloy Catalysts for CO2 Reduction Reaction [61.9212585617803]
  Graph neural networks (GNNs) have drawn more and more attention from material scientists. We develop a multi-task (MT) architecture based on DimeNet++ and mixture density networks to improve the performance of such task.
  arXiv  Detail & Related papers  (2022-09-15T13:52:15Z)
• Graph Neural Networks for Temperature-Dependent Activity Coefficient Prediction of Solutes in Ionic Liquids [58.720142291102135]
  We present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training.
  arXiv  Detail & Related papers  (2022-06-23T15:27:29Z)
• Chemical-Reaction-Aware Molecule Representation Learning [88.79052749877334]
  We propose using chemical reactions to assist learning molecule representation. Our approach is proven effective to 1) keep the embedding space well-organized and 2) improve the generalization ability of molecule embeddings. Experimental results demonstrate that our method achieves state-of-the-art performance in a variety of downstream tasks.
  arXiv  Detail & Related papers  (2021-09-21T00:08:43Z)
• Graph Neural Network Architecture Search for Molecular Property Prediction [1.0965065178451106]
  We develop an NAS approach to automate the design and development of graph neural networks (GNNs) for molecular property prediction. Specifically, we focus on automated development of message-passing neural networks (MPNNs) to predict the molecular properties of small molecules in quantum mechanics and physical chemistry data sets.
  arXiv  Detail & Related papers  (2020-08-27T15:30:57Z)
• Assessing Graph-based Deep Learning Models for Predicting Flash Point [52.931492216239995]
  Graph-based deep learning (GBDL) models were implemented in predicting flash point for the first time. Average R2 and Mean Absolute Error (MAE) scores of MPNN are, respectively, 2.3% lower and 2.0 K higher than previous comparable studies.
  arXiv  Detail & Related papers  (2020-02-26T06:10:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.