Big Data Analytics for Network Level Short-Term Travel Time Prediction
with Hierarchical LSTM
- URL: http://arxiv.org/abs/2201.05760v3
- Date: Mon, 27 Nov 2023 23:38:19 GMT
- Title: Big Data Analytics for Network Level Short-Term Travel Time Prediction
with Hierarchical LSTM
- Authors: Tianya T. Zhang
- Abstract summary: This paper utilizes a large-scale travel time dataset from Caltrans Performance Measurement System (PeMS)
To overcome the challenges of the massive amount of data, the big data analytic engines Apache Spark and Apache MXNet are applied for data wrangling and modeling.
The hierarchical LSTM model can consider the dependencies at different time scales to capture the spatial-temporal correlations of network-level travel time.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The travel time data collected from widespread traffic monitoring sensors
necessitate big data analytic tools for querying, visualization, and
identifying meaningful traffic patterns. This paper utilizes a large-scale
travel time dataset from Caltrans Performance Measurement System (PeMS) system
that is an overflow for traditional data processing and modeling tools. To
overcome the challenges of the massive amount of data, the big data analytic
engines Apache Spark and Apache MXNet are applied for data wrangling and
modeling. Seasonality and autocorrelation were performed to explore and
visualize the trend of time-varying data. Inspired by the success of the
hierarchical architecture for many Artificial Intelligent (AI) tasks, we
consolidate the cell and hidden states passed from low-level to the high-level
LSTM with an attention pooling similar to how the human perception system
operates. The designed hierarchical LSTM model can consider the dependencies at
different time scales to capture the spatial-temporal correlations of
network-level travel time. Another self-attention module is then devised to
connect LSTM extracted features to the fully connected layers, predicting
travel time for all corridors instead of a single link/route. The comparison
results show that the Hierarchical LSTM with Attention (HierLSTMat) model gives
the best prediction results at 30-minute and 45-min horizons and can
successfully forecast unusual congestion. The efficiency gained from big data
analytic tools was evaluated by comparing them with popular data science and
deep learning frameworks.
Related papers
- GATGPT: A Pre-trained Large Language Model with Graph Attention Network
for Spatiotemporal Imputation [19.371155159744934]
In real-world settings, such data often contain missing elements due to issues like sensor malfunctions and data transmission errors.
The objective oftemporal imputation is to estimate these missing values by understanding the inherent spatial and temporal relationships in the observed time series.
Traditionally, intricatetemporal imputation has relied on specific architectures, which suffer from limited applicability and high computational complexity.
In contrast our approach integrates pre-trained large language models (LLMs) into intricatetemporal imputation, introducing a groundbreaking framework, GATGPT.
arXiv Detail & Related papers (2023-11-24T08:15:11Z) - LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting [65.71129509623587]
Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning.
However, the promising results achieved on current public datasets may not be applicable to practical scenarios.
We introduce the LargeST benchmark dataset, which includes a total of 8,600 sensors in California with a 5-year time coverage.
arXiv Detail & Related papers (2023-06-14T05:48:36Z) - Multi model LSTM architecture for Track Association based on Automatic
Identification System Data [2.094022863940315]
We propose a Long Short-Term Memory (LSTM) based multi-model framework for track association.
We evaluate the performance of our approach using standard performance metrics, such as precision, recall, and F1 score.
arXiv Detail & Related papers (2023-04-04T03:11:49Z) - A CNN-LSTM Architecture for Marine Vessel Track Association Using
Automatic Identification System (AIS) Data [2.094022863940315]
This study introduces a 1D CNN-LSTM architecture-based framework for track association.
The proposed framework takes the marine vessel's location and motion data collected through the Automatic Identification System (AIS) as input and returns the most likely vessel track as output in real-time.
arXiv Detail & Related papers (2023-03-24T15:26:49Z) - Online Evolutionary Neural Architecture Search for Multivariate
Non-Stationary Time Series Forecasting [72.89994745876086]
This work presents the Online Neuro-Evolution-based Neural Architecture Search (ONE-NAS) algorithm.
ONE-NAS is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks.
Results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods.
arXiv Detail & Related papers (2023-02-20T22:25:47Z) - Automated Dilated Spatio-Temporal Synchronous Graph Modeling for Traffic
Prediction [1.6449390849183363]
We propose an automated dilated-temporal synchronous graph network prediction named Auto-DSTS for traffic prediction.
Specifically, we propose an automated dilated-temporal-temporal graph (Auto-DSTS) module to capture the short-term and long-term-temporal correlations.
Our model can achieve about 10% improvements compared with the state-of-art methods.
arXiv Detail & Related papers (2022-07-22T00:50:39Z) - Deep Cellular Recurrent Network for Efficient Analysis of Time-Series
Data with Spatial Information [52.635997570873194]
This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information.
The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
arXiv Detail & Related papers (2021-01-12T20:08:18Z) - DS-Net: Dynamic Spatiotemporal Network for Video Salient Object
Detection [78.04869214450963]
We propose a novel dynamic temporal-temporal network (DSNet) for more effective fusion of temporal and spatial information.
We show that the proposed method achieves superior performance than state-of-the-art algorithms.
arXiv Detail & Related papers (2020-12-09T06:42:30Z) - Object Tracking through Residual and Dense LSTMs [67.98948222599849]
Deep learning-based trackers based on LSTMs (Long Short-Term Memory) recurrent neural networks have emerged as a powerful alternative.
DenseLSTMs outperform Residual and regular LSTM, and offer a higher resilience to nuisances.
Our case study supports the adoption of residual-based RNNs for enhancing the robustness of other trackers.
arXiv Detail & Related papers (2020-06-22T08:20:17Z) - Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network
for Forecasting Network-wide Traffic State with Missing Values [23.504633202965376]
We focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models.
A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assist the design of neural network structures for traffic state forecasting.
We also propose a data imputation mechanism in the LSTM structure (LSTM-I) by designing an imputation unit to infer missing values and assist traffic prediction.
arXiv Detail & Related papers (2020-05-24T00:17:15Z) - Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence.
This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time.
Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
arXiv Detail & Related papers (2020-02-21T05:00:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.