Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity

• URL: http://arxiv.org/abs/2105.04854v1
• Date: Tue, 11 May 2021 08:13:34 GMT
• Title: Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity
• Authors: Ryan Henderson, Djork-Arn\'e Clevert, Floriane Montanari
• Abstract summary: We propose two simple regularization techniques to apply during the training of GCNNs. BRO encourages graph convolution operations to generate orthonormal node embeddings. Gini regularization is applied to the weights of the output layer and constrains the number of dimensions the model can use to make predictions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rationalizing which parts of a molecule drive the predictions of a molecular graph convolutional neural network (GCNN) can be difficult. To help, we propose two simple regularization techniques to apply during the training of GCNNs: Batch Representation Orthonormalization (BRO) and Gini regularization. BRO, inspired by molecular orbital theory, encourages graph convolution operations to generate orthonormal node embeddings. Gini regularization is applied to the weights of the output layer and constrains the number of dimensions the model can use to make predictions. We show that Gini and BRO regularization can improve the accuracy of state-of-the-art GCNN attribution methods on artificial benchmark datasets. In a real-world setting, we demonstrate that medicinal chemists significantly prefer explanations extracted from regularized models. While we only study these regularizers in the context of GCNNs, both can be applied to other types of neural networks

Related papers

• Learning to Reweight for Graph Neural Network [63.978102332612906]
  Graph Neural Networks (GNNs) show promising results for graph tasks. Existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data. We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability.
  arXiv  Detail & Related papers  (2023-12-19T12:25:10Z)
• ChiENN: Embracing Molecular Chirality with Graph Neural Networks [10.19088492223333]
  We propose a theoretically justified message-passing scheme, which makes GNNs sensitive to the order of node neighbors. We apply that concept in the context of molecular chirality to construct Chiral Edge Neural Network layer which can be appended to any GNN model. Our experiments show that adding ChiENN layers to a GNN outperforms current state-of-the-art methods in chiral-sensitive molecular property prediction tasks.
  arXiv  Detail & Related papers  (2023-07-05T10:50:40Z)
• OrthoReg: Improving Graph-regularized MLPs via Orthogonality Regularization [66.30021126251725]
  Graph Neural Networks (GNNs) are currently dominating in modeling graphstructure data. Graph-regularized networks (GR-MLPs) implicitly inject the graph structure information into model weights, while their performance can hardly match that of GNNs in most tasks. We show that GR-MLPs suffer from dimensional collapse, a phenomenon in which the largest a few eigenvalues dominate the embedding space. We propose OrthoReg, a novel GR-MLP model to mitigate the dimensional collapse issue.
  arXiv  Detail & Related papers  (2023-01-31T21:20:48Z)
• Graph neural networks for the prediction of molecular structure-property relationships [59.11160990637615]
  Graph neural networks (GNNs) are a novel machine learning method that directly work on the molecular graph. GNNs allow to learn properties in an end-to-end fashion, thereby avoiding the need for informative descriptors. We describe the fundamentals of GNNs and demonstrate the application of GNNs via two examples for molecular property prediction.
  arXiv  Detail & Related papers  (2022-07-25T11:30:44Z)
• Image-Like Graph Representations for Improved Molecular Property Prediction [7.119677737397071]
  We propose a new intrinsic molecular representation that bypasses the need for GNNs entirely, dubbed CubeMol. Our fixed-dimensional representation, when paired with a transformer model, exceeds the performance of state-of-the-art GNN models and provides a path for scalability.
  arXiv  Detail & Related papers  (2021-11-20T22:39:11Z)
• Molecular Graph Generation via Geometric Scattering [7.796917261490019]
  Graph neural networks (GNNs) have been used extensively for addressing problems in drug design and discovery. We propose a representation-first approach to molecular graph generation. We show that our architecture learns meaningful representations of drug datasets and provides a platform for goal-directed drug synthesis.
  arXiv  Detail & Related papers  (2021-10-12T18:00:23Z)
• GemNet: Universal Directional Graph Neural Networks for Molecules [7.484063729015126]
  We show that GNNs with directed edge embeddings and two-hop message passing are indeed universal approximators for predictions. We then leverage these insights and multiple structural improvements to propose the geometric message passing neural network (GemNet)
  arXiv  Detail & Related papers  (2021-06-02T15:44:55Z)
• Nonlinear State-Space Generalizations of Graph Convolutional Neural Networks [172.18295279061607]
  Graph convolutional neural networks (GCNNs) learn compositional representations from network data by nesting linear graph convolutions into nonlinearities. In this work, we approach GCNNs from a state-space perspective revealing that the graph convolutional module is a minimalistic linear state-space model. We show that this state update may be problematic because it is nonparametric, and depending on the graph spectrum it may explode or vanish. We propose a novel family of nodal aggregation rules that aggregate node features within a layer in a nonlinear state-space parametric fashion allowing for a better trade-off.
  arXiv  Detail & Related papers  (2020-10-27T19:48:56Z)
• Permutation-equivariant and Proximity-aware Graph Neural Networks with Stochastic Message Passing [88.30867628592112]
  Graph neural networks (GNNs) are emerging machine learning models on graphs. Permutation-equivariance and proximity-awareness are two important properties highly desirable for GNNs. We show that existing GNNs, mostly based on the message-passing mechanism, cannot simultaneously preserve the two properties. In order to preserve node proximities, we augment the existing GNNs with node representations.
  arXiv  Detail & Related papers  (2020-09-05T16:46:56Z)
• Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning via Gaussian Processes [144.6048446370369]
  Graph convolutional neural networks(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification. We propose a GP regression model via GCNs(GPGC) for graph-based semi-supervised learning. We conduct extensive experiments to evaluate GPGC and demonstrate that it outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-26T10:02:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.