Knowledge Distillation on Spatial-Temporal Graph Convolutional Network for Traffic Prediction

• URL: http://arxiv.org/abs/2401.11798v4
• Date: Tue, 24 Sep 2024 08:30:19 GMT
• Title: Knowledge Distillation on Spatial-Temporal Graph Convolutional Network for Traffic Prediction
• Authors: Mohammad Izadi, Mehran Safayani, Abdolreza Mirzaei,
• Abstract summary: We introduce a cost function designed to train a network with fewer parameters (the student) using distilled data from a complex network (the teacher) We use knowledge distillation, incorporating spatial-temporal correlations from the teacher network to enable the student to learn the complex patterns perceived by the teacher. Our method can maintain the student's accuracy close to that of the teacher, even with the retention of only 3% of network parameters.
• Score: 3.0450307343472405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient real-time traffic prediction is crucial for reducing transportation time. To predict traffic conditions, we employ a spatio-temporal graph neural network (ST-GNN) to model our real-time traffic data as temporal graphs. Despite its capabilities, it often encounters challenges in delivering efficient real-time predictions for real-world traffic data. Recognizing the significance of timely prediction due to the dynamic nature of real-time data, we employ knowledge distillation (KD) as a solution to enhance the execution time of ST-GNNs for traffic prediction. In this paper, We introduce a cost function designed to train a network with fewer parameters (the student) using distilled data from a complex network (the teacher) while maintaining its accuracy close to that of the teacher. We use knowledge distillation, incorporating spatial-temporal correlations from the teacher network to enable the student to learn the complex patterns perceived by the teacher. However, a challenge arises in determining the student network architecture rather than considering it inadvertently. To address this challenge, we propose an algorithm that utilizes the cost function to calculate pruning scores, addressing small network architecture search issues, and jointly fine-tunes the network resulting from each pruning stage using KD. Ultimately, we evaluate our proposed ideas on two real-world datasets, PeMSD7 and PeMSD8. The results indicate that our method can maintain the student's accuracy close to that of the teacher, even with the retention of only 3% of network parameters.

Related papers

• Graph Pruning Based Spatial and Temporal Graph Convolutional Network with Transfer Learning for Traffic Prediction [0.0]
  This study proposes a novel Spatial-temporal Convolutional Network (TL-GPSTGN) based on graph pruning and transfer learning framework. The results demonstrate the exceptional predictive accuracy of TL-GPSTGN on a single dataset, as well as its robust migration performance across different datasets.
  arXiv  Detail & Related papers  (2024-09-25T00:59:23Z)
• Contrastive Representation Learning for Dynamic Link Prediction in Temporal Networks [1.9389881806157312]
  We introduce a self-supervised method for learning representations of temporal networks. We propose a recurrent message-passing neural network architecture for modeling the information flow over time-respecting paths of temporal networks. The proposed method is tested on Enron, COLAB, and Facebook datasets.
  arXiv  Detail & Related papers  (2024-08-22T22:50:46Z)
• Gradient Transformation: Towards Efficient and Model-Agnostic Unlearning for Dynamic Graph Neural Networks [66.70786325911124]
  Graph unlearning has emerged as an essential tool for safeguarding user privacy and mitigating the negative impacts of undesirable data. With the increasing prevalence of DGNNs, it becomes imperative to investigate the implementation of dynamic graph unlearning. We propose an effective, efficient, model-agnostic, and post-processing method to implement DGNN unlearning.
  arXiv  Detail & Related papers  (2024-05-23T10:26:18Z)
• A novel hybrid time-varying graph neural network for traffic flow forecasting [3.6623539239888556]
  Real-time and precise traffic flow prediction is vital for the efficiency of intelligent transportation systems. Traditional graph neural networks (GNNs) are used to describe spatial correlations among traffic nodes in urban road networks. We have proposed a novel hybrid time-varying graph neural network (HTVGNN) for traffic flow prediction.
  arXiv  Detail & Related papers  (2024-01-17T07:21:36Z)
• Predicting Traffic Flow with Federated Learning and Graph Neural with Asynchronous Computations Network [0.0]
  We present a novel deep-learning method called Federated Learning and Asynchronous Graph Convolutional Networks (FLAGCN) Our framework incorporates the principles of asynchronous graph convolutional networks with federated learning to enhance accuracy and efficiency of real-time traffic flow prediction.
  arXiv  Detail & Related papers  (2024-01-05T09:36:42Z)
• Graph Convolutional Networks for Traffic Forecasting with Missing Values [0.5774786149181392]
  We propose a Graph Convolutional Network model with the ability to handle the complex missing values in the Spatio-temporal context. We propose as well a dynamic graph learning module based on the learned local-global features. The experimental results on real-life datasets show the reliability of our proposed method.
  arXiv  Detail & Related papers  (2022-12-13T08:04:38Z)
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach [76.45949280328838]
  We propose a Laplacian enhanced low-rank tensor (LETC) framework featuring both lowrankness and multi-temporal correlations for large-scale traffic speed kriging. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging.
  arXiv  Detail & Related papers  (2022-10-21T07:25:57Z)
• Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer [58.6106391721944]
  Cross-city knowledge has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities. We propose a model-agnostic few-shot learning framework for S-temporal graph called ST-GFSL. We conduct comprehensive experiments on four traffic speed prediction benchmarks and the results demonstrate the effectiveness of ST-GFSL compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-27T12:46:52Z)
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations [54.05212871508062]
  We propose a novel approach for dynamic network representation learning based on Temporal Graph Network. For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings. We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
  arXiv  Detail & Related papers  (2021-08-19T15:39:52Z)
• Spatio-Temporal Graph Scattering Transform [54.52797775999124]
  Graph neural networks may be impractical in some real-world scenarios due to a lack of sufficient high-quality training data. We put forth a novel mathematically designed framework to analyze-temporal data.
  arXiv  Detail & Related papers  (2020-12-06T19:49:55Z)
• Constructing Geographic and Long-term Temporal Graph for Traffic Forecasting [88.5550074808201]
  We propose Geographic and Long term Temporal Graph Convolutional Recurrent Neural Network (GLT-GCRNN) for traffic forecasting. In this work, we propose a novel framework for traffic forecasting that learns the rich interactions between roads sharing similar geographic or longterm temporal patterns.
  arXiv  Detail & Related papers  (2020-04-23T03:50:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.