A Parameter-free Nonconvex Low-rank Tensor Completion Model for Spatiotemporal Traffic Data Recovery

• URL: http://arxiv.org/abs/2209.13786v1
• Date: Wed, 28 Sep 2022 02:29:34 GMT
• Title: A Parameter-free Nonconvex Low-rank Tensor Completion Model for Spatiotemporal Traffic Data Recovery
• Authors: Yang He, Yuheng Jia, Liyang Hu, Chengchuan An, Zhenbo Lu and Jingxin Xia
• Abstract summary: Traffic data chronically suffer from missing corruption, leading to accuracy and utility reduction in subsequentITS applications. The proposed method outperforms other methods in both missing corrupted data recovery.
• Score: 14.084532939272766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic data chronically suffer from missing and corruption, leading to accuracy and utility reduction in subsequent Intelligent Transportation System (ITS) applications. Noticing the inherent low-rank property of traffic data, numerous studies formulated missing traffic data recovery as a low-rank tensor completion (LRTC) problem. Due to the non-convexity and discreteness of the rank minimization in LRTC, existing methods either replaced rank with convex surrogates that are quite far away from the rank function or approximated rank with nonconvex surrogates involving many parameters. In this study, we proposed a Parameter-Free Non-Convex Tensor Completion model (TC-PFNC) for traffic data recovery, in which a log-based relaxation term was designed to approximate tensor algebraic rank. Moreover, previous studies usually assumed the observations are reliable without any outliers. Therefore, we extended the TC-PFNC to a robust version (RTC-PFNC) by modeling potential traffic data outliers, which can recover the missing value from partial and corrupted observations and remove the anomalies in observations. The numerical solutions of TC-PFNC and RTC-PFNC were elaborated based on the alternating direction multiplier method (ADMM). The extensive experimental results conducted on four real-world traffic data sets demonstrated that the proposed methods outperform other state-of-the-art methods in both missing and corrupted data recovery. The code used in this paper is available at: https://github.com/YoungHe49/T-ITSPFNC.

Related papers

• Machine Learning Techniques for Data Reduction of CFD Applications [10.881548113461493]
  We present an approach called guaranteed block autoencoder that leverages Correlations for reducing scientific results. It uses a multidimensional block of tensors (CFD) for both input and output.
  arXiv  Detail & Related papers  (2024-04-28T04:01:09Z)
• Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
  The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization. We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
  arXiv  Detail & Related papers  (2023-10-09T14:30:06Z)
• Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity [60.791736094073]
  Communication overhead has become one of the major bottlenecks in the distributed training of deep neural networks. We propose a magnitude-driven sparsification scheme, which addresses the non-convergence issue of SIGNSGD. The proposed scheme is validated through experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2023-02-19T17:42:35Z)
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation [151.70234052015948]
  We propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7%.
  arXiv  Detail & Related papers  (2022-11-20T15:49:11Z)
• Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patterns [77.34726150561087]
  We introduce four complicated missing patterns, including missing and three fiber-like missing cases according to the mode-drivenn fibers. Despite nonity of the objective function in our model, we derive the optimal solutions by integrating alternating data-mputation method of multipliers.
  arXiv  Detail & Related papers  (2022-05-19T08:37:56Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Low-Rank Hankel Tensor Completion for Traffic Speed Estimation [7.346671461427793]
  We propose a purely data-driven and model-free solution to the traffic state estimation problem. By imposing a low-rank assumption on this tensor structure, we can approximate characterize both global patterns and the unknown complex local dynamics. We conduct numerical experiments on both synthetic simulation data and real-world high-resolution data, and our results demonstrate the effectiveness and superiority of the proposed model.
  arXiv  Detail & Related papers  (2021-05-21T00:08:06Z)
• Low-Rank Autoregressive Tensor Completion for Spatiotemporal Traffic Data Imputation [4.9831085918734805]
  Missing data imputation has been a long-standing research topic and critical application for real-world intelligent transportation systems. We propose a low-rank autoregressive tensor completion (LATC) framework by introducing textittemporal variation as a new regularization term. We conduct extensive numerical experiments on several real-world traffic data sets, and our results demonstrate the effectiveness of LATC in diverse missing scenarios.
  arXiv  Detail & Related papers  (2021-04-30T12:00:57Z)
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso [102.84915019938413]
  Non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG) offer promise of non-invasive techniques. The problem of source localization, or source imaging, poses however a high-dimensional statistical inference challenge. We propose an ensemble of desparsified multi-task Lasso (ecd-MTLasso) to deal with this problem.
  arXiv  Detail & Related papers  (2020-09-29T21:17:16Z)
• A Nonconvex Low-Rank Tensor Completion Model for Spatiotemporal Traffic Data Imputation [13.48205738743634]
  Missing data imputation is common in intemporal traffic data collected from various sensing systems. We present an efficient algorithm to obtain the optimal solution for each variable. The proposed model also outperforms other baseline models in extreme missing scenarios.
  arXiv  Detail & Related papers  (2020-03-23T13:27:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.