Modeling Information Blackouts in Missing Not-At-Random Time Series Data

• URL: http://arxiv.org/abs/2601.01480v2
• Date: Tue, 06 Jan 2026 10:56:56 GMT
• Title: Modeling Information Blackouts in Missing Not-At-Random Time Series Data
• Abstract summary: Large-scale traffic forecasting relies on fixed sensor networks that often exhibit blackouts.<n>We propose a latent state-space framework that jointly models traffic dynamics via a linear dynamical system.<n>Explicit MNAR modeling provides a consistent but smaller additional improvement on real data.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale traffic forecasting relies on fixed sensor networks that often exhibit blackouts: contiguous intervals of missing measurements caused by detector or communication failures. These outages are typically handled under a Missing At Random (MAR) assumption, even though blackout events may correlate with unobserved traffic conditions (e.g., congestion or anomalous flow), motivating a Missing Not At Random (MNAR) treatment. We propose a latent state-space framework that jointly models (i) traffic dynamics via a linear dynamical system and (ii) sensor dropout via a Bernoulli observation channel whose probability depends on the latent traffic state. Inference uses an Extended Kalman Filter with Rauch-Tung-Striebel smoothing, and parameters are learned via an approximate EM procedure with a dedicated update for detector-specific missingness parameters. On the Seattle inductive loop detector data, introducing latent dynamics yields large gains over naive baselines, reducing blackout imputation RMSE from 7.02 (LOCF) and 5.02 (linear interpolation + seasonal naive) to 4.23 (MAR LDS), corresponding to about a 64% reduction in MSE relative to LOCF. Explicit MNAR modeling provides a consistent but smaller additional improvement on real data (imputation RMSE 4.20; 0.8% RMSE reduction relative to MAR), with similar modest gains for short-horizon post-blackout forecasts (evaluated at 1, 3, and 6 steps). In controlled synthetic experiments, the MNAR advantage increases as the true missingness dependence on latent state strengthens. Overall, temporal dynamics dominate performance, while MNAR modeling offers a principled refinement that becomes most valuable when missingness is genuinely informative.

Related papers

• A Multi-View Coupled Tensor Decomposition for Lightweight Online Adaptive Traffic Prediction [9.202409218074294]
  This paper proposes a Multi-View Coupled Decomposition (MVCTD) model for online traffic prediction from imperfect multi-view observations.<n>Experiments on real-world traffic datasets demonstrate that MVCTD achieves accurate forecasts with favorable runtime under severe missingness.
  arXiv  Detail & Related papers  (2026-08-26T08:09:00Z)
• Metropolis-Scale Resilient and Trustworthy Traffic Flow Inference Using Multi-Source Data [20.65202203037699]
  Task-Aware Attentive Neural Process (TA-ANP) is a unified framework for trustworthy global traffic state inference (GTSI)<n>For uncertainty quantification, we combine neural processes with Monte Carlo Dropout to capture both aleatoric and uncertainty.<n>Experiments show TA-ANP achieves state-of-the-art performance across all sub-tasks under deterministic and probabilistic metrics.
  arXiv  Detail & Related papers  (2026-05-24T11:12:02Z)
• Unifying Sign and Magnitude for Optimizing Deep Vision Networks via ThermoLion [0.0]
  Current paradigms impose a static compromise on information channel drift parameters.<n>We introduce a "low-dimensional" exploration model and a "low-dimensional" dynamic alignment framework.
  arXiv  Detail & Related papers  (2025-12-01T17:04:17Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving [64.42138266293202]
  ResAD is a Normalized Residual Trajectory Modeling framework.<n>It reframes the learning task to predict the residual deviation from an inertial reference.<n>On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy.
  arXiv  Detail & Related papers  (2025-10-09T17:59:36Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• A Hybrid PCA-PR-Seq2Seq-Adam-LSTM Framework for Time-Series Power Outage Prediction [5.657115189763182]
  This paper introduces a hybrid deep learning framework, termed PCA-PR-Seq2Seq-Adam-LSTM.<n>It integrates Principal Component Analysis (PCA), Poisson Regression (PR), a Sequence-to-Sequence (Seq2Seq) architecture, and an Adam-optimized LSTM.<n>Results indicate that the proposed approach significantly improves forecasting accuracy and robustness compared to existing methods.
  arXiv  Detail & Related papers  (2025-09-20T17:13:25Z)
• Can the Waymo Open Motion Dataset Support Realistic Behavioral Modeling? A Validation Study with Naturalistic Trajectories [4.250936281629339]
  This study examines whether WOMD accurately captures the dynamics and interactions observed in real-world AV operations.<n>We perform comparative analyses across three representative urban driving scenarios: discharging at signalized intersections, car-following, and lane-changing behaviors.<n>Results consistently show that behavior in PHX falls outside the behavioral envelope of WOMD.
  arXiv  Detail & Related papers  (2025-09-03T17:56:46Z)
• A Simple Approximate Bayesian Inference Neural Surrogate for Stochastic Petri Net Models [0.0]
  We introduce a neural-network-based approximation of the posterior distribution framework.<n>Our model employs a lightweight 1D Convolutional Residual Network trained end-to-end on Gillespie-simulated SPN realizations.<n>On synthetic SPNs with 20% missing events, our surrogate recovers rate-function coefficients with an RMSE = 0.108 and substantially runs faster than traditional Bayesian approaches.
  arXiv  Detail & Related papers  (2025-07-14T18:31:19Z)
• STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation [36.880711201508085]
  Existing time-to-space methods often fail to effectively extract features in block-wise missing data scenarios.<n>This paper proposes a Spatiotemporal Attention Mixture of experts network named STAMImputer for traffic data imputation.<n>The result shows STAMImputer achieves significantly performance improvement compared with existing SOTA approaches.
  arXiv  Detail & Related papers  (2025-06-09T04:05:00Z)
• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• Time-inversion of spatiotemporal beam dynamics using uncertainty-aware latent evolution reversal [46.348283638884425]
  This paper introduces a reverse Latent Evolution Model (rLEM) designed for temporal phase of forward beam dynamics. In this two-step self-supervised deep learning framework, we utilize a Conditional Autoencoder (CVAE) to project 6D space projections of a charged particle beam into a lower-dimensional latent distribution. We then autoregressively learn the inverse temporal dynamics in the latent space using a Long Short-Term Memory (LSTM) network.
  arXiv  Detail & Related papers  (2024-08-14T23:09:01Z)
• DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detection [52.74152717667157]
  We propose a lightweight module called Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops.
  arXiv  Detail & Related papers  (2024-06-05T06:18:03Z)
• Towards better traffic volume estimation: Jointly addressing the underdetermination and nonequilibrium problems with correlation-adaptive GNNs [47.18837782862979]
  This paper studies two key problems with regard to traffic volume estimation: (1) underdetermined traffic flows caused by undetected movements, and (2) non-equilibrium traffic flows arise from congestion propagation. We demonstrate a graph-based deep learning method that can offer a data-driven, model-free and correlation adaptive approach to tackle the above issues.
  arXiv  Detail & Related papers  (2023-03-10T02:22:33Z)
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach [76.45949280328838]
  We propose a Laplacian enhanced low-rank tensor (LETC) framework featuring both lowrankness and multi-temporal correlations for large-scale traffic speed kriging. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging.
  arXiv  Detail & Related papers  (2022-10-21T07:25:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.