Integrating Temporal and Structural Context in Graph Transformers for Relational Deep Learning

• URL: http://arxiv.org/abs/2511.04557v1
• Date: Thu, 06 Nov 2025 17:08:21 GMT
• Title: Integrating Temporal and Structural Context in Graph Transformers for Relational Deep Learning
• Authors: Divyansha Lachi, Mahmoud Mohammadi, Joe Meyer, Vinam Arora, Tom Palczewski, Eva L. Dyer,
• Abstract summary: We introduce a temporal subgraph sampler that enhances global context by retrieving nodes beyond the immediate neighborhood.<n>We also propose Graph Perceiver (RGP), a graph transformer architecture for relational deep learning.<n>RGP delivers state-of-the-art performance, offering a general and scalable solution for relational deep learning.
• Score: 6.780460717949915
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In domains such as healthcare, finance, and e-commerce, the temporal dynamics of relational data emerge from complex interactions-such as those between patients and providers, or users and products across diverse categories. To be broadly useful, models operating on these data must integrate long-range spatial and temporal dependencies across diverse types of entities, while also supporting multiple predictive tasks. However, existing graph models for relational data primarily focus on spatial structure, treating temporal information merely as a filtering constraint to exclude future events rather than a modeling signal, and are typically designed for single-task prediction. To address these gaps, we introduce a temporal subgraph sampler that enhances global context by retrieving nodes beyond the immediate neighborhood to capture temporally relevant relationships. In addition, we propose the Relational Graph Perceiver (RGP), a graph transformer architecture for relational deep learning that leverages a cross-attention-based latent bottleneck to efficiently integrate information from both structural and temporal contexts. This latent bottleneck integrates signals from different node and edge types into a common latent space, enabling the model to build global context across the entire relational system. RGP also incorporates a flexible cross-attention decoder that supports joint learning across tasks with disjoint label spaces within a single model. Experiments on RelBench, SALT, and CTU show that RGP delivers state-of-the-art performance, offering a general and scalable solution for relational deep learning with support for diverse predictive tasks.

Related papers

• Unlocking Dynamic Inter-Client Spatial Dependencies: A Federated Spatio-Temporal Graph Learning Method for Traffic Flow Forecasting [13.035721364418471]
  Federated Spatio-Temporal Graph with Dynamic Inter-Client Dependencies (FedSTGD) is a framework designed to model and reconstruct dynamic inter-client spatial dependencies in federated learning.<n>Experiments on four real-world datasets demonstrate that FedSTGD achieves superior performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-13T15:57:19Z)
• Hybrid Hypergraph Networks for Multimodal Sequence Data Classification [9.688069013427057]
  We propose the hybrid hypergraph network (HHN), a novel framework that models temporal multimodal data via a segmentation-first, graph-later strategy.<n>HHN achieves state-of-the-art results on four multimodal datasets, demonstrating its effectiveness in complex classification tasks.
  arXiv  Detail & Related papers  (2025-07-30T12:13:05Z)
• A Pre-training Framework for Relational Data with Information-theoretic Principles [57.93973948947743]
  We introduce Task Vector Estimation (TVE), a novel pre-training framework that constructs supervisory signals via set-based aggregation over relational graphs.<n>TVE consistently outperforms traditional pre-training baselines.<n>Our findings advocate for pre-training objectives that encode task heterogeneity and temporal structure as design principles for predictive modeling on relational databases.
  arXiv  Detail & Related papers  (2025-07-14T00:17:21Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures [50.46688111973999]
  Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data.<n>We present a new blueprint that enables end-to-end representation of'relational entity graphs' without traditional engineering feature.<n>We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data.
  arXiv  Detail & Related papers  (2025-06-19T23:51:38Z)
• STRGCN: Capturing Asynchronous Spatio-Temporal Dependencies for Irregular Multivariate Time Series Forecasting [14.156419219696252]
  STRGCN captures the complex interdependencies in IMTS by representing them as a fully connected graph.<n>Experiments on four public datasets demonstrate that STRGCN achieves state-of-the-art accuracy, competitive memory usage and training speed.
  arXiv  Detail & Related papers  (2025-05-07T06:41:33Z)
• Explaining Categorical Feature Interactions Using Graph Covariance and LLMs [18.44675735926458]
  This paper focuses on the global synthetic dataset from the Counter Trafficking Data Collaborative.<n>It contains over 200,000 anonymized records spanning from 2002 to 2022 with numerous categorical features for each record.<n>We propose a fast and scalable method for analyzing and extracting significant categorical feature interactions.
  arXiv  Detail & Related papers  (2025-01-24T21:41:26Z)
• Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation [64.85974098314344]
  Video scene graph generation (VidSGG) aims to identify objects in visual scenes and infer their relationships for a given video. Inherently, object pairs and their relationships enjoy spatial co-occurrence correlations within each image and temporal consistency/transition correlations across different images. We propose a spatial-temporal knowledge-embedded transformer (STKET) that incorporates the prior spatial-temporal knowledge into the multi-head cross-attention mechanism.
  arXiv  Detail & Related papers  (2023-09-23T02:40:28Z)
• Multi-Temporal Relationship Inference in Urban Areas [75.86026742632528]
  Finding temporal relationships among locations can benefit a bunch of urban applications, such as dynamic offline advertising and smart public transport planning. We propose a solution to Trial with a graph learning scheme, which includes a spatially evolving graph neural network (SEENet) SEConv performs the intra-time aggregation and inter-time propagation to capture the multifaceted spatially evolving contexts from the view of location message passing. SE-SSL designs time-aware self-supervised learning tasks in a global-local manner with additional evolving constraint to enhance the location representation learning and further handle the relationship sparsity.
  arXiv  Detail & Related papers  (2023-06-15T07:48:32Z)
• An Adaptive Federated Relevance Framework for Spatial Temporal Graph Learning [14.353798949041698]
  We propose an adaptive federated relevance framework, namely FedRel, for spatial-temporal graph learning. The core Dynamic Inter-Intra Graph (DIIG) module in the framework is able to use these features to generate the spatial-temporal graphs. To improve the model generalization ability and performance while preserving the local data privacy, we also design a relevance-driven federated learning module.
  arXiv  Detail & Related papers  (2022-06-07T16:12:17Z)
• Dynamic Relation Discovery and Utilization in Multi-Entity Time Series Forecasting [92.32415130188046]
  In many real-world scenarios, there could exist crucial yet implicit relation between entities. We propose an attentional multi-graph neural network with automatic graph learning (A2GNN) in this work.
  arXiv  Detail & Related papers  (2022-02-18T11:37:04Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.