Knowledge Distillation on Spatial-Temporal Graph Convolutional Network
for Traffic Prediction
- URL: http://arxiv.org/abs/2401.11798v3
- Date: Sun, 28 Jan 2024 06:00:23 GMT
- Title: Knowledge Distillation on Spatial-Temporal Graph Convolutional Network
for Traffic Prediction
- Authors: Mohammad Izadi, Mehran Safayani, Abdolreza Mirzaei
- Abstract summary: We introduce a cost function designed to train a network with fewer parameters (the student) using distilled data from a complex network (the teacher)
We use knowledge distillation, incorporating spatial-temporal correlations from the teacher network to enable the student to learn the complex patterns perceived by the teacher.
Our method can maintain the student's accuracy close to that of the teacher, even with the retention of only $3%$ of network parameters.
- Score: 3.422309388045879
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Efficient real-time traffic prediction is crucial for reducing transportation
time. To predict traffic conditions, we employ a spatio-temporal graph neural
network (ST-GNN) to model our real-time traffic data as temporal graphs.
Despite its capabilities, it often encounters challenges in delivering
efficient real-time predictions for real-world traffic data. Recognizing the
significance of timely prediction due to the dynamic nature of real-time data,
we employ knowledge distillation (KD) as a solution to enhance the execution
time of ST-GNNs for traffic prediction. In this paper, We introduce a cost
function designed to train a network with fewer parameters (the student) using
distilled data from a complex network (the teacher) while maintaining its
accuracy close to that of the teacher. We use knowledge distillation,
incorporating spatial-temporal correlations from the teacher network to enable
the student to learn the complex patterns perceived by the teacher. However, a
challenge arises in determining the student network architecture rather than
considering it inadvertently. To address this challenge, we propose an
algorithm that utilizes the cost function to calculate pruning scores,
addressing small network architecture search issues, and jointly fine-tunes the
network resulting from each pruning stage using KD. Ultimately, we evaluate our
proposed ideas on two real-world datasets, PeMSD7 and PeMSD8. The results
indicate that our method can maintain the student's accuracy close to that of
the teacher, even with the retention of only $3\%$ of network parameters.
Related papers
- Gradient Transformation: Towards Efficient and Model-Agnostic Unlearning for Dynamic Graph Neural Networks [66.70786325911124]
Graph unlearning has emerged as an essential tool for safeguarding user privacy and mitigating the negative impacts of undesirable data.
With the increasing prevalence of DGNNs, it becomes imperative to investigate the implementation of dynamic graph unlearning.
We propose an effective, efficient, model-agnostic, and post-processing method to implement DGNN unlearning.
arXiv Detail & Related papers (2024-05-23T10:26:18Z) - A novel hybrid time-varying graph neural network for traffic flow forecasting [3.6623539239888556]
Real-time and precise traffic flow prediction is vital for the efficiency of intelligent transportation systems.
Traditional graph neural networks (GNNs) are used to describe spatial correlations among traffic nodes in urban road networks.
We have proposed a novel hybrid time-varying graph neural network (HTVGNN) for traffic flow prediction.
arXiv Detail & Related papers (2024-01-17T07:21:36Z) - Predicting Traffic Flow with Federated Learning and Graph Neural with Asynchronous Computations Network [0.0]
We present a novel deep-learning method called Federated Learning and Asynchronous Graph Convolutional Networks (FLAGCN)
Our framework incorporates the principles of asynchronous graph convolutional networks with federated learning to enhance accuracy and efficiency of real-time traffic flow prediction.
arXiv Detail & Related papers (2024-01-05T09:36:42Z) - Attention-based Spatial-Temporal Graph Neural ODE for Traffic Prediction [3.4806267677524896]
We propose attention-based graph neural ODE (AST) that explicitly learns the dynamics of the traffic system.
Our model aggregates traffic patterns of different periods and has satisfactory performance on two real-world traffic data sets.
arXiv Detail & Related papers (2023-05-01T00:58:48Z) - Graph Convolutional Networks for Traffic Forecasting with Missing Values [0.5774786149181392]
We propose a Graph Convolutional Network model with the ability to handle the complex missing values in the Spatio-temporal context.
We propose as well a dynamic graph learning module based on the learned local-global features.
The experimental results on real-life datasets show the reliability of our proposed method.
arXiv Detail & Related papers (2022-12-13T08:04:38Z) - Efficient Traffic State Forecasting using Spatio-Temporal Network
Dependencies: A Sparse Graph Neural Network Approach [6.203371866342754]
Traffic prediction in a transportation network is paramount for effective traffic operations and management.
Long-term traffic prediction (beyond 30 minutes into the future) remains challenging in current research.
We propose sparse training to the training cost, while preserving the prediction accuracy.
arXiv Detail & Related papers (2022-11-06T05:41:39Z) - Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices.
The impact of on-device storage on the performance of FL is still not explored.
In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
arXiv Detail & Related papers (2022-09-01T03:27:33Z) - Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge
Transfer [58.6106391721944]
Cross-city knowledge has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities.
We propose a model-agnostic few-shot learning framework for S-temporal graph called ST-GFSL.
We conduct comprehensive experiments on four traffic speed prediction benchmarks and the results demonstrate the effectiveness of ST-GFSL compared with state-of-the-art methods.
arXiv Detail & Related papers (2022-05-27T12:46:52Z) - Temporal Graph Network Embedding with Causal Anonymous Walks
Representations [54.05212871508062]
We propose a novel approach for dynamic network representation learning based on Temporal Graph Network.
For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings.
We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
arXiv Detail & Related papers (2021-08-19T15:39:52Z) - Online learning of windmill time series using Long Short-term Cognitive
Networks [58.675240242609064]
The amount of data generated on windmill farms makes online learning the most viable strategy to follow.
We use Long Short-term Cognitive Networks (LSTCNs) to forecast windmill time series in online settings.
Our approach reported the lowest forecasting errors with respect to a simple RNN, a Long Short-term Memory, a Gated Recurrent Unit, and a Hidden Markov Model.
arXiv Detail & Related papers (2021-07-01T13:13:24Z) - Constructing Geographic and Long-term Temporal Graph for Traffic
Forecasting [88.5550074808201]
We propose Geographic and Long term Temporal Graph Convolutional Recurrent Neural Network (GLT-GCRNN) for traffic forecasting.
In this work, we propose a novel framework for traffic forecasting that learns the rich interactions between roads sharing similar geographic or longterm temporal patterns.
arXiv Detail & Related papers (2020-04-23T03:50:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.