GRAMA: Adaptive Graph Autoregressive Moving Average Models

• URL: http://arxiv.org/abs/2501.12732v1
• Date: Wed, 22 Jan 2025 09:09:17 GMT
• Title: GRAMA: Adaptive Graph Autoregressive Moving Average Models
• Authors: Moshe Eliasof, Alessio Gravina, Andrea Ceni, Claudio Gallicchio, Davide Bacciu, Carola-Bibiane Schönlieb,
• Abstract summary: We introduce GRAMA, a Graph Adaptive method based on a learnable Autoregressive Moving Average (ARMA) framework.<n>By transforming from static to sequential graph data, GRAMA enables efficient and flexible long-range information propagation.<n>We also establish theoretical connections between GRAMA and Selective SSMs, providing insights into its ability to capture long-range dependencies.
• Score: 26.755971450887333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph State Space Models (SSMs) have recently been introduced to enhance Graph Neural Networks (GNNs) in modeling long-range interactions. Despite their success, existing methods either compromise on permutation equivariance or limit their focus to pairwise interactions rather than sequences. Building on the connection between Autoregressive Moving Average (ARMA) and SSM, in this paper, we introduce GRAMA, a Graph Adaptive method based on a learnable Autoregressive Moving Average (ARMA) framework that addresses these limitations. By transforming from static to sequential graph data, GRAMA leverages the strengths of the ARMA framework, while preserving permutation equivariance. Moreover, GRAMA incorporates a selective attention mechanism for dynamic learning of ARMA coefficients, enabling efficient and flexible long-range information propagation. We also establish theoretical connections between GRAMA and Selective SSMs, providing insights into its ability to capture long-range dependencies. Extensive experiments on 14 synthetic and real-world datasets demonstrate that GRAMA consistently outperforms backbone models and performs competitively with state-of-the-art methods.

Related papers

• Global-Lens Transformers: Adaptive Token Mixing for Dynamic Link Prediction [9.234363752442915]
  We propose GLFormer, a novel attention-free Transformer-style framework for dynamic graphs.<n>Experiments on six widely-used dynamic graph benchmarks show that GLFormer achieves SOTA performance, which reveals that attention-free architectures can match or surpass Transformer baselines.
  arXiv  Detail & Related papers  (2025-11-16T04:05:56Z)
• GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning [50.40400074353263]
  Graph Neural Networks (GNNs) are powerful tools for precessing relational data but often struggle to generalize to unseen graphs.<n>We introduce textbfGraph textbfIn-context textbfL textbfTransformer (GILT), a framework built on an LLM-free and tuning-free architecture.
  arXiv  Detail & Related papers  (2025-10-06T08:09:15Z)
• G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge [88.82814893945077]
  Large language models (LLMs) excel at complex reasoning but remain limited by static and incomplete parametric knowledge.<n>Recent graph-enhanced RAG (GraphRAG) attempts to bridge this gap by constructing tailored graphs and enabling LLMs to reason on them.<n>G-reasoner is a unified framework that integrates graph and language foundation models for reasoning over diverse graph-structured knowledge.
  arXiv  Detail & Related papers  (2025-09-29T04:38:12Z)
• LLM-based Contrastive Self-Supervised AMR Learning with Masked Graph Autoencoders for Fake News Detection [17.045049022252563]
  The proliferation of misinformation in the digital age has led to significant societal challenges.<n>Existing approaches often struggle with capturing long-range dependencies, complex semantic relations, and the social dynamics influencing news dissemination.<n>We propose a novel self-supervised misinformation detection framework that integrates both complex semantic relations and news propagation dynamics.
  arXiv  Detail & Related papers  (2025-08-26T08:58:35Z)
• EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation [50.848374648774374]
  MultiModal Recommendation (MMR) systems have emerged as a promising solution for improving recommendation quality by leveraging rich item-side modality information.<n>We propose EGRA, which incorporates into the behavior graph an item-item graph built from representations generated by a pretrained MMR model.<n>It also introduces a novel bi-level dynamic alignment weighting mechanism to improve modality-behavior representation alignment.
  arXiv  Detail & Related papers  (2025-08-22T07:47:54Z)
• Message-Passing State-Space Models: Improving Graph Learning with Modern Sequence Modeling [19.10832920407789]
  We introduce a new perspective by embedding the key principles of modern SSM directly into the Message-Passing Neural Network framework.<n>Our approach, MP-SSM, enables efficient, permutation-equivariant, and long-range information propagation while preserving the architectural simplicity of message passing.
  arXiv  Detail & Related papers  (2025-05-24T14:53:07Z)
• Text-Derived Relational Graph-Enhanced Network for Skeleton-Based Action Segmentation [14.707224594220264]
  We propose a Text-Derived Graph Network (TRG-Net) to enhance both modeling and supervision. For modeling, the Dynamic Spatio-Temporal Fusion Modeling (D) method incorporates Text-Derived Joint Graphs (JGT) with channel adaptation. For supervision, the Absolute-Relative Inter-Class Supervision (ARIS) method employs contrastive learning between action features and text embeddings to regularize the absolute class.
  arXiv  Detail & Related papers  (2025-03-19T11:38:14Z)
• Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers [0.0]
  We reformulate the Transformer's attention mechanism as a graph operation.<n>We introduce Sparse GIN-Attention, a fine-tuning approach that employs sparse GINs.
  arXiv  Detail & Related papers  (2025-01-04T22:30:21Z)
• A Comparative Study on Dynamic Graph Embedding based on Mamba and Transformers [0.29687381456164]
  This study presents a comparative analysis of dynamic graph embedding approaches using transformers and the recently proposed Mamba architecture.<n>We introduce three novel models: TransformerG2G augment with graph convolutional networks, DG-Mamba, and GDG-Mamba with graph isomorphism network edge convolutions.<n>Our experiments on multiple benchmark datasets demonstrate that Mamba-based models achieve comparable or superior performance to transformer-based approaches in link prediction tasks.
  arXiv  Detail & Related papers  (2024-12-15T19:56:56Z)
• LLM-Based Multi-Agent Systems are Scalable Graph Generative Models [73.28294528654885]
  GraphAgent-Generator (GAG) is a novel simulation-based framework for dynamic, text-attributed social graph generation.<n>GAG simulates the temporal node and edge generation processes for zero-shot social graph generation.<n>The resulting graphs exhibit adherence to seven key macroscopic network properties, achieving an 11% improvement in microscopic graph structure metrics.
  arXiv  Detail & Related papers  (2024-10-13T12:57:08Z)
• Retrieval Augmented Generation for Dynamic Graph Modeling [15.09162213134372]
  We propose a novel framework, Retrieval-Augmented Generation for Dynamic Graph modeling (RAG4DyG) RAG4DyG enhances dynamic graph predictions by incorporating contextually and temporally relevant examples from broader graph structures. The proposed framework is designed to be effective in both transductive and inductive scenarios.
  arXiv  Detail & Related papers  (2024-08-26T09:23:35Z)
• DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs [59.434893231950205]
  Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
  arXiv  Detail & Related papers  (2024-08-13T15:21:46Z)
• Sparse Graphical Linear Dynamical Systems [1.6635799895254402]
  Time-series datasets are central in machine learning with applications in numerous fields of science and engineering. This work proposes a novel approach to bridge the gap by introducing a joint graphical modeling framework. We present DGLASSO, a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm.
  arXiv  Detail & Related papers  (2023-07-06T14:10:02Z)
• Data-heterogeneity-aware Mixing for Decentralized Learning [63.83913592085953]
  We characterize the dependence of convergence on the relationship between the mixing weights of the graph and the data heterogeneity across nodes. We propose a metric that quantifies the ability of a graph to mix the current gradients. Motivated by our analysis, we propose an approach that periodically and efficiently optimize the metric.
  arXiv  Detail & Related papers  (2022-04-13T15:54:35Z)
• TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
  We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion. To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
  arXiv  Detail & Related papers  (2021-05-17T15:33:25Z)
• Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation [56.73834525802723]
  Lightweight Dynamic Graph Convolutional Networks (LDGCNs) are proposed. LDGCNs capture richer non-local interactions by synthesizing higher order information from the input graphs. We develop two novel parameter saving strategies based on the group graph convolutions and weight tied convolutions to reduce memory usage and model complexity.
  arXiv  Detail & Related papers  (2020-10-09T06:03:46Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.