Multi-Task Edge Prediction in Temporally-Dynamic Video Graphs

• URL: http://arxiv.org/abs/2212.02875v1
• Date: Tue, 6 Dec 2022 10:41:00 GMT
• Title: Multi-Task Edge Prediction in Temporally-Dynamic Video Graphs
• Authors: Osman \"Ulger, Julian Wiederer, Mohsen Ghafoorian, Vasileios Belagiannis, Pascal Mettes
• Abstract summary: We propose MTD-GNN, a graph network for predicting temporally-dynamic edges for multiple types of relations. We show that modeling multiple relations in our temporal-dynamic graph network can be mutually beneficial.
• Score: 16.121140184388786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph neural networks have shown to learn effective node representations, enabling node-, link-, and graph-level inference. Conventional graph networks assume static relations between nodes, while relations between entities in a video often evolve over time, with nodes entering and exiting dynamically. In such temporally-dynamic graphs, a core problem is inferring the future state of spatio-temporal edges, which can constitute multiple types of relations. To address this problem, we propose MTD-GNN, a graph network for predicting temporally-dynamic edges for multiple types of relations. We propose a factorized spatio-temporal graph attention layer to learn dynamic node representations and present a multi-task edge prediction loss that models multiple relations simultaneously. The proposed architecture operates on top of scene graphs that we obtain from videos through object detection and spatio-temporal linking. Experimental evaluations on ActionGenome and CLEVRER show that modeling multiple relations in our temporally-dynamic graph network can be mutually beneficial, outperforming existing static and spatio-temporal graph neural networks, as well as state-of-the-art predicate classification methods.

Related papers

• Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatio-temporal Forecasting [60.03169701753824]
  We propose a novel Dynamic Diffusion-al Graph Neural Network (DVGNN) fortemporal forecasting. The proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding Root Mean Squared Error result.
  arXiv  Detail & Related papers  (2023-05-16T11:38:19Z)
• Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation [67.26422477327179]
  Temporal graphs exhibit dynamic interactions between nodes over continuous time. We propose a novel method of temporal graph convolution with the whole neighborhood. Our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency.
  arXiv  Detail & Related papers  (2023-04-15T08:17:18Z)
• TodyNet: Temporal Dynamic Graph Neural Network for Multivariate Time Series Classification [6.76723360505692]
  We propose a novel temporal dynamic neural graph network (TodyNet) that can extract hidden-temporal dependencies without undefined graph structure. The experiments on 26 UEA benchmark datasets illustrate that the proposed TodyNet outperforms existing deep learning-based methods in the MTSC tasks.
  arXiv  Detail & Related papers  (2023-04-11T09:21:28Z)
• Time-aware Dynamic Graph Embedding for Asynchronous Structural Evolution [60.695162101159134]
  Existing works merely view a dynamic graph as a sequence of changes. We formulate dynamic graphs as temporal edge sequences associated with joining time of. vertex and timespan of edges. A time-aware Transformer is proposed to embed. vertex' dynamic connections and ToEs into the learned. vertex representations.
  arXiv  Detail & Related papers  (2022-07-01T15:32:56Z)
• Learning the Evolutionary and Multi-scale Graph Structure for Multivariate Time Series Forecasting [50.901984244738806]
  We show how to model the evolutionary and multi-scale interactions of time series. In particular, we first provide a hierarchical graph structure cooperated with the dilated convolution to capture the scale-specific correlations. A unified neural network is provided to integrate the components above to get the final prediction.
  arXiv  Detail & Related papers  (2022-06-28T08:11:12Z)
• Continuous Temporal Graph Networks for Event-Based Graph Data [41.786721257905555]
  We propose Continuous Temporal Graph Networks (CTGNs) to capture the continuous dynamics of temporal graph data. Key idea is to use neural ordinary differential equations (ODE) to characterize the continuous dynamics of node representations over dynamic graphs. Experiment results on both transductive and inductive tasks demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2022-05-31T16:17:02Z)
• Deep Dynamic Effective Connectivity Estimation from Multivariate Time Series [0.0]
  We develop dynamic effective connectivity estimation via neural network training (DECENNT) DECENNT outperforms state-of-the-art (SOTA) methods on five different tasks and infers interpretable task-specific dynamic graphs.
  arXiv  Detail & Related papers  (2022-02-04T21:14:21Z)
• Dynamic Graph Learning-Neural Network for Multivariate Time Series Modeling [2.3022070933226217]
  We propose a novel framework, namely static- and dynamic-graph learning-neural network (GL) The model acquires static and dynamic graph matrices from data to model long-term and short-term patterns respectively. It achieves state-of-the-art performance on almost all datasets.
  arXiv  Detail & Related papers  (2021-12-06T08:19:15Z)
• Spatio-Temporal Joint Graph Convolutional Networks for Traffic Forecasting [75.10017445699532]
  Recent have shifted their focus towards formulating traffic forecasting as atemporal graph modeling problem. We propose a novel approach for accurate traffic forecasting on road networks over multiple future time steps.
  arXiv  Detail & Related papers  (2021-11-25T08:45:14Z)
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations [54.05212871508062]
  We propose a novel approach for dynamic network representation learning based on Temporal Graph Network. For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings. We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
  arXiv  Detail & Related papers  (2021-08-19T15:39:52Z)
• Temporal Relational Modeling with Self-Supervision for Action Segmentation [38.62057004624234]
  We introduce Dilated Temporal Graph Reasoning Module (DTGRM) to model temporal relations in video. In particular, we capture and model temporal relations via constructing multi-level dilated temporal graphs. Our model outperforms state-of-the-art action segmentation models on three challenging datasets.
  arXiv  Detail & Related papers  (2020-12-14T13:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.