Missing Value Imputation for Multi-attribute Sensor Data Streams via Message Propagation (Extended Version)

• URL: http://arxiv.org/abs/2311.07344v2
• Date: Tue, 14 Nov 2023 14:39:58 GMT
• Title: Missing Value Imputation for Multi-attribute Sensor Data Streams via Message Propagation (Extended Version)
• Authors: Xiao Li, Huan Li, Hua Lu, Christian S. Jensen, Varun Pandey, and Volker Markl
• Abstract summary: We propose a message propagation imputation network (MPIN) that is able to recover the missing values of data instances in a time window. MPIN can outperform the existing data imputers by wide margins and that the continuous imputation framework is efficient and accurate.
• Score: 25.022656067909523
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Sensor data streams occur widely in various real-time applications in the context of the Internet of Things (IoT). However, sensor data streams feature missing values due to factors such as sensor failures, communication errors, or depleted batteries. Missing values can compromise the quality of real-time analytics tasks and downstream applications. Existing imputation methods either make strong assumptions about streams or have low efficiency. In this study, we aim to accurately and efficiently impute missing values in data streams that satisfy only general characteristics in order to benefit real-time applications more widely. First, we propose a message propagation imputation network (MPIN) that is able to recover the missing values of data instances in a time window. We give a theoretical analysis of why MPIN is effective. Second, we present a continuous imputation framework that consists of data update and model update mechanisms to enable MPIN to perform continuous imputation both effectively and efficiently. Extensive experiments on multiple real datasets show that MPIN can outperform the existing data imputers by wide margins and that the continuous imputation framework is efficient and accurate.

Related papers

• SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio [17.811771707446926]
  We show that learning based methods can, even based on synthetic data, significantly outperform GCC-PHAT on novel real world data. We provide our trained model, SONNET, which is runnable in real-time and works on novel data out of the box for many real data applications.
  arXiv  Detail & Related papers  (2024-11-20T10:23:21Z)
• Real-time Monitoring of Lower Limb Movement Resistance Based on Deep Learning [0.0]
  Real-time lower limb movement resistance monitoring is critical for various applications in clinical and sports settings, such as rehabilitation and athletic training. We propose a novel Mobile Multi-Task Learning Network (MMTL-Net) that integrates MobileNetV3 for efficient feature extraction and employs multi-task learning to simultaneously predict resistance levels and recognize activities. The advantages of MMTL-Net include enhanced accuracy, reduced latency, and improved computational efficiency, making it highly suitable for real-time applications.
  arXiv  Detail & Related papers  (2024-10-13T18:19:48Z)
• PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection [65.24854366973794]
  Node-level graph anomaly detection (GAD) plays a critical role in identifying anomalous nodes from graph-structured data in domains such as medicine, social networks, and e-commerce. We introduce a simple method termed PREprocessing and Matching (PREM for short) to improve the efficiency of GAD. Our approach streamlines GAD, reducing time and memory consumption while maintaining powerful anomaly detection capabilities.
  arXiv  Detail & Related papers  (2023-10-18T02:59:57Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)
• HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse [49.1501082763252]
  This paper presents HFEDMS for incorporating practical FL into the emerging Industrial Metaverse. It reduces data heterogeneity through dynamic grouping and training mode conversion. Then, it compensates for the forgotten knowledge by fusing compressed historical data semantics. Experiments have been conducted on the streamed non-i.i.d. FEMNIST dataset using 368 simulated devices.
  arXiv  Detail & Related papers  (2022-11-07T04:33:24Z)
• SECOE: Alleviating Sensors Failure in Machine Learning-Coupled IoT Systems [0.0]
  This paper proposes SECOE, a proactive approach for alleviating potentially simultaneous sensor failures. SECOE includes a novel technique to minimize the number of models in the ensemble by harnessing the correlations among sensors. Experiments reveal that SECOE effectively preserves prediction accuracy in the presence of sensor failures.
  arXiv  Detail & Related papers  (2022-10-05T10:58:39Z)
• DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness [78.98998551326812]
  We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis. We show that the method can exploit the multi-modality features from related sensors and also learn from history time-series dynamics to reconstruct the data under extreme missingness.
  arXiv  Detail & Related papers  (2022-09-26T21:59:14Z)
• STING: Self-attention based Time-series Imputation Networks using GAN [4.052758394413726]
  STING (Self-attention based Time-series Imputation Networks using GAN) is proposed. We take advantage of generative adversarial networks and bidirectional recurrent neural networks to learn latent representations of the time series. Experimental results on three real-world datasets demonstrate that STING outperforms the existing state-of-the-art methods in terms of imputation accuracy.
  arXiv  Detail & Related papers  (2022-09-22T06:06:56Z)
• Development of fully intuitionistic fuzzy data envelopment analysis model with missing data: an application to Indian police sector [2.9434930072968584]
  DEA is a technique used to measure the efficiency of decision-making units (DMUs) Data is usually collected by humans, machines, or both. A method is presented that can deal with missing values and inaccuracy in the data. A real-life application to measure the performance efficiencies of Indian police stations is presented.
  arXiv  Detail & Related papers  (2022-07-27T18:20:13Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Transfer Learning for Fault Diagnosis of Transmission Lines [55.971052290285485]
  A novel transfer learning framework based on a pre-trained LeNet-5 convolutional neural network is proposed. It is able to diagnose faults for different transmission line lengths and impedances by transferring the knowledge from a source neural network to predict a dissimilar target dataset.
  arXiv  Detail & Related papers  (2022-01-20T06:36:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.