Symplectic Structure-Aware Hamiltonian (Graph) Embeddings

• URL: http://arxiv.org/abs/2309.04885v4
• Date: Tue, 23 Jul 2024 20:10:42 GMT
• Title: Symplectic Structure-Aware Hamiltonian (Graph) Embeddings
• Authors: Jiaxu Liu, Xinping Yi, Tianle Zhang, Xiaowei Huang,
• Abstract summary: Hamiltonian system-inspired GNNs have been proposed to address the dynamic nature of such embeddings. We present Symplectic Structure-Aware Hamiltonian GNN (SAH-GNN), a novel approach that generalizes Hamiltonian dynamics for more flexible node feature updates.
• Score: 21.0714383010908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In traditional Graph Neural Networks (GNNs), the assumption of a fixed embedding manifold often limits their adaptability to diverse graph geometries. Recently, Hamiltonian system-inspired GNNs have been proposed to address the dynamic nature of such embeddings by incorporating physical laws into node feature updates. We present Symplectic Structure-Aware Hamiltonian GNN (SAH-GNN), a novel approach that generalizes Hamiltonian dynamics for more flexible node feature updates. Unlike existing Hamiltonian approaches, SAH-GNN employs Riemannian optimization on the symplectic Stiefel manifold to adaptively learn the underlying symplectic structure, circumventing the limitations of existing Hamiltonian GNNs that rely on a pre-defined form of standard symplectic structure. This innovation allows SAH-GNN to automatically adapt to various graph datasets without extensive hyperparameter tuning. Moreover, it conserves energy during training meaning the implicit Hamiltonian system is physically meaningful. Finally, we empirically validate SAH-GNN's superiority and adaptability in node classification tasks across multiple types of graph datasets.

Related papers

• Spatiotemporal Learning on Cell-embedded Graphs [6.8090864965073274]
  We introduce a learnable cell attribution to the node-edge message passing process, which better captures the spatial dependency of regional features. Experiments on various PDE systems and one real-world dataset demonstrate that CeGNN achieves superior performance compared with other baseline models.
  arXiv  Detail & Related papers  (2024-09-26T16:22:08Z)
• Graph Neural Reaction Diffusion Models [14.164952387868341]
  We propose a novel family of Reaction GNNs based on neural RD systems. We discuss the theoretical properties of our RDGNN, its implementation, and show that it improves or offers competitive performance to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-16T09:46:58Z)
• DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing [72.70197960100677]
  Graph Structure Learning (GSL) plays an important role in boosting Graph Neural Networks (GNNs) with a refined graph. GSL solutions usually focus on structure refinement with task-specific supervision (i.e., node classification) or overlook the inherent weakness of GNNs themselves. We propose to study self-supervised graph structure-feature co-refinement for effectively alleviating the issue of over-squashing in typical GNNs.
  arXiv  Detail & Related papers  (2024-01-23T14:06:08Z)
• Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph [58.99478502486377]
  We propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections.
  arXiv  Detail & Related papers  (2022-09-23T05:24:18Z)
• MGNN: Graph Neural Networks Inspired by Distance Geometry Problem [28.789684784093048]
  Graph Neural Networks (GNNs) have emerged as a prominent research topic in the field of machine learning. In this paper, we propose a GNN model inspired by the congruent-inphilic property of the classifiers in the classification phase of GNNs. We extensively evaluate the effectiveness of our model through experiments conducted on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-01-31T04:15:42Z)
• Continuous-Depth Neural Models for Dynamic Graph Prediction [16.89981677708299]
  We introduce the framework of continuous-depth graph neural networks (GNNs) Neural graph differential equations (Neural GDEs) are formalized as the counterpart to GNNs. Results prove the effectiveness of the proposed models across applications, such as traffic forecasting or prediction in genetic regulatory networks.
  arXiv  Detail & Related papers  (2021-06-22T07:30:35Z)
• A unified framework for Hamiltonian deep neural networks [3.0934684265555052]
  Training deep neural networks (DNNs) can be difficult due to vanishing/exploding gradients during weight optimization. We propose a class of DNNs stemming from the time discretization of Hamiltonian systems. The proposed Hamiltonian framework, besides encompassing existing networks inspired by marginally stable ODEs, allows one to derive new and more expressive architectures.
  arXiv  Detail & Related papers  (2021-04-27T13:20:24Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)
• A Unified View on Graph Neural Networks as Graph Signal Denoising [49.980783124401555]
  Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data. In this work, we establish mathematically that the aggregation processes in a group of representative GNN models can be regarded as solving a graph denoising problem. We instantiate a novel GNN model, ADA-UGNN, derived from UGNN, to handle graphs with adaptive smoothness across nodes.
  arXiv  Detail & Related papers  (2020-10-05T04:57:18Z)
• Permutation-equivariant and Proximity-aware Graph Neural Networks with Stochastic Message Passing [88.30867628592112]
  Graph neural networks (GNNs) are emerging machine learning models on graphs. Permutation-equivariance and proximity-awareness are two important properties highly desirable for GNNs. We show that existing GNNs, mostly based on the message-passing mechanism, cannot simultaneously preserve the two properties. In order to preserve node proximities, we augment the existing GNNs with node representations.
  arXiv  Detail & Related papers  (2020-09-05T16:46:56Z)
• Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs [95.63153473559865]
  Graph Neural Networks (GNNs) are emerging machine learning models on graphs. Most existing GNN models in practice are shallow and essentially feature-centric. We show empirically and analytically that the existing shallow GNNs cannot preserve graph structures well. We propose Eigen-GNN, a plug-in module to boost GNNs ability in preserving graph structures.
  arXiv  Detail & Related papers  (2020-06-08T02:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.