Making Graph Neural Networks Worth It for Low-Data Molecular Machine Learning

• URL: http://arxiv.org/abs/2011.12203v1
• Date: Tue, 24 Nov 2020 16:52:04 GMT
• Title: Making Graph Neural Networks Worth It for Low-Data Molecular Machine Learning
• Authors: Aneesh Pappu, Brooks Paige
• Abstract summary: We investigate whether graph neural networks are competitive in small data settings compared to the parametrically 'cheaper' alternative of fingerprint methods. We find that MAML and FO-MAML do enable the graph neural network to outperform models based on fingerprints. In contrast to previous work, we find ANIL performs worse that other meta-learning approaches in this molecule setting.
• Score: 15.251466525698627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph neural networks have become very popular for machine learning on molecules due to the expressive power of their learnt representations. However, molecular machine learning is a classically low-data regime and it isn't clear that graph neural networks can avoid overfitting in low-resource settings. In contrast, fingerprint methods are the traditional standard for low-data environments due to their reduced number of parameters and manually engineered features. In this work, we investigate whether graph neural networks are competitive in small data settings compared to the parametrically 'cheaper' alternative of fingerprint methods. When we find that they are not, we explore pretraining and the meta-learning method MAML (and variants FO-MAML and ANIL) for improving graph neural network performance by transfer learning from related tasks. We find that MAML and FO-MAML do enable the graph neural network to outperform models based on fingerprints, providing a path to using graph neural networks even in settings with severely restricted data availability. In contrast to previous work, we find ANIL performs worse that other meta-learning approaches in this molecule setting. Our results suggest two reasons: molecular machine learning tasks may require significant task-specific adaptation, and distribution shifts in test tasks relative to train tasks may contribute to worse ANIL performance.

Related papers

• Neural Graph Matching Improves Retrieval Augmented Generation in Molecular Machine Learning [20.911425911621865]
  We introduce MARASON, a novel model that incorporates neural graph matching to enhance a fragmentation-based neural network. MarASON achieves 28% top-1 accuracy, a substantial improvement over the non-retrieval state-of-the-art accuracy of 19%.
  arXiv  Detail & Related papers  (2025-02-25T05:59:14Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Graph Metanetworks for Processing Diverse Neural Architectures [33.686728709734105]
  Graph Metanetworks (GMNs) generalizes to neural architectures where competing methods struggle. We prove that GMNs are expressive and equivariant to parameter permutation symmetries that leave the input neural network functions.
  arXiv  Detail & Related papers  (2023-12-07T18:21:52Z)
• Unlearning Graph Classifiers with Limited Data Resources [39.29148804411811]
  Controlled data removal is becoming an important feature of machine learning models for data-sensitive Web applications. It is still largely unknown how to perform efficient machine unlearning of graph neural networks (GNNs) Our main contribution is the first known nonlinear approximate graph unlearning method based on GSTs. Our second contribution is a theoretical analysis of the computational complexity of the proposed unlearning mechanism. Our third contribution are extensive simulation results which show that, compared to complete retraining of GNNs after each removal request, the new GST-based approach offers, on average, a 10.38x speed-up
  arXiv  Detail & Related papers  (2022-11-06T20:46:50Z)
• Graph Neural Networks with Trainable Adjacency Matrices for Fault Diagnosis on Multivariate Sensor Data [69.25738064847175]
  It is necessary to consider the behavior of the signals in each sensor separately, to take into account their correlation and hidden relationships with each other. The graph nodes can be represented as data from the different sensors, and the edges can display the influence of these data on each other. It was proposed to construct a graph during the training of graph neural network. This allows to train models on data where the dependencies between the sensors are not known in advance.
  arXiv  Detail & Related papers  (2022-10-20T11:03:21Z)
• Can we learn gradients by Hamiltonian Neural Networks? [68.8204255655161]
  We propose a meta-learner based on ODE neural networks that learns gradients. We demonstrate that our method outperforms a meta-learner based on LSTM for an artificial task and the MNIST dataset with ReLU activations in the optimizee.
  arXiv  Detail & Related papers  (2021-10-31T18:35:10Z)
• Training Graph Neural Networks by Graphon Estimation [2.5997274006052544]
  We propose to train a graph neural network via resampling from a graphon estimate obtained from the underlying network data. We show that our approach is competitive with and in many cases outperform the other over-smoothing reducing GNN training methods.
  arXiv  Detail & Related papers  (2021-09-04T19:21:48Z)
• Weakly-supervised Graph Meta-learning for Few-shot Node Classification [53.36828125138149]
  We propose a new graph meta-learning framework -- Graph Hallucination Networks (Meta-GHN) Based on a new robustness-enhanced episodic training, Meta-GHN is meta-learned to hallucinate clean node representations from weakly-labeled data. Extensive experiments demonstrate the superiority of Meta-GHN over existing graph meta-learning studies.
  arXiv  Detail & Related papers  (2021-06-12T22:22:10Z)
• Variational models for signal processing with Graph Neural Networks [3.5939555573102853]
  This paper is devoted to signal processing on point-clouds by means of neural networks. In this work, we investigate the use of variational models for such Graph Neural Networks to process signals on graphs for unsupervised learning.
  arXiv  Detail & Related papers  (2021-03-30T13:31:11Z)
• E(n) Equivariant Graph Neural Networks [86.75170631724548]
  This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs) In contrast with existing methods, our work does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance.
  arXiv  Detail & Related papers  (2021-02-19T10:25:33Z)
• From Boltzmann Machines to Neural Networks and Back Again [31.613544605376624]
  We give new results for learning Restricted Boltzmann Machines, probably the most well-studied class of latent variable models. Our results are based on new connections to learning two-layer neural networks under $ell_infty$ bounded input. We then give an algorithm for learning a natural class of supervised RBMs with better runtime than what is possible for its related class of networks without distributional assumptions.
  arXiv  Detail & Related papers  (2020-07-25T00:42:50Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.