Molecule Property Prediction and Classification with Graph Hypernetworks

• URL: http://arxiv.org/abs/2002.00240v1
• Date: Sat, 1 Feb 2020 16:44:34 GMT
• Title: Molecule Property Prediction and Classification with Graph Hypernetworks
• Authors: Eliya Nachmani, Lior Wolf
• Abstract summary: We show that the replacement of the underlying networks with hypernetworks leads to a boost in performance. A major difficulty in the application of hypernetworks is their lack of stability. A recent work has tackled the training instability of hypernetworks in the context of error correcting codes.
• Score: 113.38181979662288
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph neural networks are currently leading the performance charts in learning-based molecule property prediction and classification. Computational chemistry has, therefore, become the a prominent testbed for generic graph neural networks, as well as for specialized message passing methods. In this work, we demonstrate that the replacement of the underlying networks with hypernetworks leads to a boost in performance, obtaining state of the art results in various benchmarks. A major difficulty in the application of hypernetworks is their lack of stability. We tackle this by combining the current message and the first message. A recent work has tackled the training instability of hypernetworks in the context of error correcting codes, by replacing the activation function of the message passing network with a low-order Taylor approximation of it. We demonstrate that our generic solution can replace this domain-specific solution.

Related papers

• Online Learning Of Expanding Graphs [14.952056744888916]
  This paper addresses the problem of online network inference for expanding graphs from a stream of signals. We introduce a strategy that enables different types of updates for nodes that just joined the network and for previously existing nodes.
  arXiv  Detail & Related papers  (2024-09-13T09:20:42Z)
• Formal Verification of Graph Convolutional Networks with Uncertain Node Features and Uncertain Graph Structure [7.133681867718039]
  Graph neural networks are becoming increasingly popular in the field of machine learning. They have been applied in safety-critical environments where perturbations inherently occur. This research addresses the non-passing gap by preserving the dependencies of all elements in the underlying computations.
  arXiv  Detail & Related papers  (2024-04-23T14:12:48Z)
• GNN-LoFI: a Novel Graph Neural Network through Localized Feature-based Histogram Intersection [51.608147732998994]
  Graph neural networks are increasingly becoming the framework of choice for graph-based machine learning. We propose a new graph neural network architecture that substitutes classical message passing with an analysis of the local distribution of node features.
  arXiv  Detail & Related papers  (2024-01-17T13:04:23Z)
• Centered Self-Attention Layers [89.21791761168032]
  The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied. We show that this application inevitably leads to oversmoothing, i.e., to similar representations at the deeper layers. We present a correction term to the aggregating operator of these mechanisms.
  arXiv  Detail & Related papers  (2023-06-02T15:19:08Z)
• Magnitude Invariant Parametrizations Improve Hypernetwork Learning [0.0]
  Hypernetworks are powerful neural networks that predict the parameters of another neural network. Training typically converges far more slowly than for non-hypernetwork models. We identify a fundamental and previously unidentified problem that contributes to the challenge of training hypernetworks. We present a simple solution to this problem using a revised hypernetwork formulation that we call Magnitude Invariant Parametrizations (MIP)
  arXiv  Detail & Related papers  (2023-04-15T22:18:29Z)
• Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning [0.0]
  We introduce a copy node' method that successfully initializes arbitrary tensor networks. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models.
  arXiv  Detail & Related papers  (2022-03-03T19:00:40Z)
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations [54.05212871508062]
  We propose a novel approach for dynamic network representation learning based on Temporal Graph Network. For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings. We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
  arXiv  Detail & Related papers  (2021-08-19T15:39:52Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.