Learning graph geometry and topology using dynamical systems based message-passing

• URL: http://arxiv.org/abs/2309.09924v4
• Date: Sun, 7 Jul 2024 23:08:05 GMT
• Title: Learning graph geometry and topology using dynamical systems based message-passing
• Authors: Dhananjay Bhaskar, Yanlei Zhang, Charles Xu, Xingzhi Sun, Oluwadamilola Fasina, Guy Wolf, Maximilian Nickel, Michael Perlmutter, Smita Krishnaswamy,
• Abstract summary: We introduce DYMAG: a message passing paradigm for GNNs built on the expressive power of graph-dynamics. DYMAG makes use of complex graph dynamics based on the heat and wave equation as well as a more complex equation which admits chaotic solutions. We demonstrate that DYMAG achieves superior performance in recovering the generating parameters of Erd"os-Renyi and block random graphs.
• Score: 21.571006438656323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we introduce DYMAG: a message passing paradigm for GNNs built on the expressive power of continuous, multiscale graph-dynamics. Standard discrete-time message passing algorithms implicitly make use of simplistic graph dynamics and aggregation schemes which limit their ability to capture fundamental graph topological properties. By contrast, DYMAG makes use of complex graph dynamics based on the heat and wave equation as well as a more complex equation which admits chaotic solutions. The continuous nature of the dynamics are leveraged to generate multiscale (dynamic-time snapshot) representations which we prove are linked to various graph topological and spectral properties. We demonstrate experimentally that DYMAG achieves superior performance in recovering the generating parameters of Erd\"os-Renyi and stochastic block model random graphs and the persistent homology of synthetic graphs and citation network. Since the behavior of proteins and biomolecules is sensitive to graph topology and exhibits important structure at multiple scales, we find that DYMAG outperforms other methods at predicting salient features of various biomolecules.

Related papers

• Graph Wave Networks [17.80926325018177]
  We develop a graph wave equation to leverage the wave propagation on graphs.<n>In details, we demonstrate that the graph wave equation can be connected to traditional spectral GNNs.<n>Experiments show that GWNs achieve SOTA and efficient performance on benchmark datasets.
  arXiv  Detail & Related papers  (2025-05-26T14:20:41Z)
• A Geometry-Aware Message Passing Neural Network for Modeling Aerodynamics over Airfoils [61.60175086194333]
  aerodynamics is a key problem in aerospace engineering, often involving flows interacting with solid objects such as airfoils.<n>Here, we consider modeling of incompressible flows over solid objects, wherein geometric structures are a key factor in determining aerodynamics.<n>To effectively incorporate geometries, we propose a message passing scheme that efficiently and expressively integrates the airfoil shape with the mesh representation.<n>These design choices lead to a purely data-driven machine learning framework known as GeoMPNN, which won the Best Student Submission award at the NeurIPS 2024 ML4CFD Competition, placing 4th overall.
  arXiv  Detail & Related papers  (2024-12-12T16:05:39Z)
• PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation [37.35829410807451]
  We propose PeriodWave, a novel universal waveform generation model. We introduce a period-aware flow matching estimator that can capture the periodic features of the waveform signal. We also propose a single period-conditional universal estimator that can feed-forward parallel by period-wise batch inference.
  arXiv  Detail & Related papers  (2024-08-14T13:36:17Z)
• State Space Models on Temporal Graphs: A First-Principles Study [30.531930200222423]
  Research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. We develop GraphSSM, a graph state space model for modeling the dynamics of temporal graphs.
  arXiv  Detail & Related papers  (2024-06-03T02:56:11Z)
• Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputation [9.450743095412896]
  Missing values is an essential but challenging issue due to the complex latent-temporal correlation and dynamic nature of time series. We propose a higher-ordertemporal physics-incorporated Graph Neural Networks (HSPGNN) to address this problem. HSPGNN provides better dynamic analysis and explanation than traditional data-driven models.
  arXiv  Detail & Related papers  (2024-05-16T16:35:43Z)
• Learning Coarse-Grained Dynamics on Graph [4.692217705215042]
  We consider a Graph Neural Network (GNN) non-Markovian modeling framework to identify coarse-grained dynamical systems on graphs. Our main idea is to systematically determine the GNN architecture by inspecting how the leading term of the Mori-Zwanzig memory term depends on the coarse-grained interaction coefficients that encode the graph topology.
  arXiv  Detail & Related papers  (2024-05-15T13:25:34Z)
• Advective Diffusion Transformers for Topological Generalization in Graph Learning [69.2894350228753]
  We show how graph diffusion equations extrapolate and generalize in the presence of varying graph topologies. We propose a novel graph encoder backbone, Advective Diffusion Transformer (ADiT), inspired by advective graph diffusion equations.
  arXiv  Detail & Related papers  (2023-10-10T08:40:47Z)
• Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatio-temporal Forecasting [60.03169701753824]
  We propose a novel Dynamic Diffusion-al Graph Neural Network (DVGNN) fortemporal forecasting. The proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding Root Mean Squared Error result.
  arXiv  Detail & Related papers  (2023-05-16T11:38:19Z)
• Dynamic Graph Representation Learning via Edge Temporal States Modeling and Structure-reinforced Transformer [5.093187534912688]
  We introduce the Recurrent Structure-reinforced Graph Transformer (RSGT), a novel framework for dynamic graph representation learning. RSGT captures temporal node representations encoding both graph topology and evolving dynamics through a recurrent learning paradigm. We show RSGT's superior performance in discrete dynamic graph representation learning, consistently outperforming existing methods in dynamic link prediction tasks.
  arXiv  Detail & Related papers  (2023-04-20T04:12:50Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations [21.936437653875245]
  This paper focuses on representation learning for dynamic graphs with temporal interactions. We propose a generic differential model for dynamic graphs that characterises the continuously dynamic evolution of node embedding trajectories. Our framework exhibits several desirable characteristics, including the ability to express dynamics on evolving graphs without integration by segments.
  arXiv  Detail & Related papers  (2023-02-22T12:59:38Z)
• Graph Generation with Diffusion Mixture [57.78958552860948]
  Generation of graphs is a major challenge for real-world tasks that require understanding the complex nature of their non-Euclidean structures. We propose a generative framework that models the topology of graphs by explicitly learning the final graph structures of the diffusion process.
  arXiv  Detail & Related papers  (2023-02-07T17:07:46Z)
• Continuous Temporal Graph Networks for Event-Based Graph Data [41.786721257905555]
  We propose Continuous Temporal Graph Networks (CTGNs) to capture the continuous dynamics of temporal graph data. Key idea is to use neural ordinary differential equations (ODE) to characterize the continuous dynamics of node representations over dynamic graphs. Experiment results on both transductive and inductive tasks demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2022-05-31T16:17:02Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs [77.33781731432163]
  We learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer node representations. We present a novel Hyperbolic Variational Graph Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach.
  arXiv  Detail & Related papers  (2021-04-06T01:44:15Z)
• Graph-Time Convolutional Neural Networks [9.137554315375919]
  We represent spatial relationships through product graphs with a first principle graph-time convolutional neural network (GTCNN) We develop a graph-time convolutional filter by following the shift-and-sumtemporal operator to learn higher-level features over the product graph. We develop a zero-pad pooling that preserves the spatial graph while reducing the number of active nodes and the parameters.
  arXiv  Detail & Related papers  (2021-03-02T14:03:44Z)
• Disentangled Dynamic Graph Deep Generation [10.934180735890727]
  This paper proposes a novel framework of factorized deep generative models to achieve interpretable dynamic graph generation. Various generative models are proposed to characterize conditional independence among node, edge, static, and dynamic factors. Experiments on multiple datasets demonstrate the effectiveness of the proposed models.
  arXiv  Detail & Related papers  (2020-10-14T17:52:49Z)
• Data-Driven Learning of Geometric Scattering Networks [74.3283600072357]
  We propose a new graph neural network (GNN) module based on relaxations of recently proposed geometric scattering transforms. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations.
  arXiv  Detail & Related papers  (2020-10-06T01:20:27Z)
• Structural Landmarking and Interaction Modelling: on Resolution Dilemmas in Graph Classification [50.83222170524406]
  We study the intrinsic difficulty in graph classification under the unified concept of resolution dilemmas'' We propose SLIM'', an inductive neural network model for Structural Landmarking and Interaction Modelling.
  arXiv  Detail & Related papers  (2020-06-29T01:01:42Z)
• Learning to Simulate Complex Physics with Graph Networks [68.43901833812448]
  We present a machine learning framework and model implementation that can learn to simulate a wide variety of challenging physical domains. Our framework---which we term "Graph Network-based Simulators" (GNS)--represents the state of a physical system with particles, expressed as nodes in a graph, and computes dynamics via learned message-passing. Our results show that our model can generalize from single-timestep predictions with thousands of particles during training, to different initial conditions, thousands of timesteps, and at least an order of magnitude more particles at test time.
  arXiv  Detail & Related papers  (2020-02-21T16:44:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.