Related papers: A Causal Convolutional Low-rank Representation Model for Imputation of Water Quality Data

A Causal Convolutional Low-rank Representation Model for Imputation of Water Quality Data

URL: http://arxiv.org/abs/2504.15209v1
Date: Mon, 21 Apr 2025 16:27:16 GMT
Title: A Causal Convolutional Low-rank Representation Model for Imputation of Water Quality Data
Authors: Xin Liao, Bing Yang, Tan Dongli, Cai Yu,
Abstract summary: This paper proposes a Causal convolutional Low-rank Representation (CLR) model for imputing missing WQD to improve the completeness of the WQD.<n> Experimental studies on three real-world water quality datasets demonstrate that the proposed CLR model is superior to some of the existing state-of-the-art imputation models.
Score: 11.584987653534531
License: http://creativecommons.org/licenses/by/4.0/
Abstract: The monitoring of water quality is a crucial part of environmental protection, and a large number of monitors are widely deployed to monitor water quality. Due to unavoidable factors such as data acquisition breakdowns, sensors and communication failures, water quality monitoring data suffers from missing values over time, resulting in High-Dimensional and Sparse (HDS) Water Quality Data (WQD). The simple and rough filling of the missing values leads to inaccurate results and affects the implementation of relevant measures. Therefore, this paper proposes a Causal convolutional Low-rank Representation (CLR) model for imputing missing WQD to improve the completeness of the WQD, which employs a two-fold idea: a) applying causal convolutional operation to consider the temporal dependence of the low-rank representation, thus incorporating temporal information to improve the imputation accuracy; and b) implementing a hyperparameters adaptation scheme to automatically adjust the best hyperparameters during model training, thereby reducing the tedious manual adjustment of hyper-parameters. Experimental studies on three real-world water quality datasets demonstrate that the proposed CLR model is superior to some of the existing state-of-the-art imputation models in terms of imputation accuracy and time cost, as well as indicating that the proposed model provides more reliable decision support for environmental monitoring.

Related papers

Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
We introduce denoising score distillation (DSD), a surprisingly effective and novel approach for training high-quality generative models from low-quality data.<n>DSD pretrains a diffusion model exclusively on noisy, corrupted samples and then distills it into a one-step generator capable of producing refined, clean outputs.
arXiv Detail & Related papers (2025-03-10T17:44:46Z)
Water Quality Data Imputation via A Fast Latent Factorization of Tensors with PID-based Optimizer [21.261626027956737]
There are numerous missing values in water quality data due to sensor failure.<n>A Latent Factorization of PIDs (LFT) with Gradient Descent (SGD) proves to be an efficient imputation method.<n>This paper proposes a Fast Latent Factorization of PIDs (FLFT) model to tackle this issue.
arXiv Detail & Related papers (2025-03-10T07:22:54Z)
AutoML for Multi-Class Anomaly Compensation of Sensor Drift [44.63945828405864]
Sensor drift degrades the performance of machine learning models over time.<n>Standard cross-validation method overestimates performance by inadequately accounting for drift.<n>This paper presents two solutions: (1) a novel sensor drift compensation learning paradigm for validating models, and (2) automated machine learning (AutoML) techniques to enhance classification performance and compensate sensor drift.
arXiv Detail & Related papers (2025-02-26T14:34:53Z)
Can Deep Learning Trigger Alerts from Mobile-Captured Images? [0.0594961162060159]
This research contributes to verification of data augmentation techniques, CNN-based regression modelling for air quality prediction, and user-centric air quality monitoring through mobile technology.<n>The proposed system offers practical solutions for individuals to make informed environmental health and well-being decisions.
arXiv Detail & Related papers (2025-01-07T03:39:43Z)
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
arXiv Detail & Related papers (2024-03-20T04:58:03Z)
Causal Disentanglement Hidden Markov Model for Fault Diagnosis [55.90917958154425]
We propose a Causal Disentanglement Hidden Markov model (CDHM) to learn the causality in the bearing fault mechanism. Specifically, we make full use of the time-series data and progressively disentangle the vibration signal into fault-relevant and fault-irrelevant factors. To expand the scope of the application, we adopt unsupervised domain adaptation to transfer the learned disentangled representations to other working environments.
arXiv Detail & Related papers (2023-08-06T05:58:45Z)
Short-term prediction of stream turbidity using surrogate data and a meta-model approach [0.0]
We build and compare the ability of dynamic regression (ARIMA), long short-term memory neural nets (LSTM), and generalized additive models (GAM) to forecast stream turbidity. We construct a meta-model, trained on time-series features of turbidity, to take advantage of the strengths of each model over different time points. Our findings indicate that temperature and light-associated variables, for example underwater illuminance, may hold promise as cost-effective surrogates of turbidity.
arXiv Detail & Related papers (2022-10-11T23:05:32Z)
Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping [58.50342759993186]
We propose a visuo-tactile model for realtime estimation of the liquid inside a deformable container. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor. The robotic system is well controlled and adjusted based on the estimation model in real time.
arXiv Detail & Related papers (2022-02-23T13:38:31Z)
SWAT Watershed Model Calibration using Deep Learning [0.860255319568951]
We present a fast, accurate, and reliable methodology to calibrate the SWAT model using deep learning (DL) We develop DL-enabled inverse models based on convolutional neural networks to ingest streamflow data and estimate the SWAT model parameters. Our results show that the DL models-based calibration is better than traditional parameter estimation methods.
arXiv Detail & Related papers (2021-10-06T22:56:23Z)
Detection of Anomalies in a Time Series Data using InfluxDB and Python [0.0]
This paper demonstrates data cleaning and preparation for time-series data. It further proposes cost-sensitive machine learning algorithms as a solution to detect anomalous data points in time-series data.
arXiv Detail & Related papers (2020-12-15T17:27:39Z)
From Rain Generation to Rain Removal [67.71728610434698]
We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator. We employ the variational inference framework to approximate the expected statistical distribution of rainy image. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
arXiv Detail & Related papers (2020-08-08T18:56:51Z)
SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
arXiv Detail & Related papers (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.