Sequence Prediction Under Missing Data : An RNN Approach Without Imputation

• URL: http://arxiv.org/abs/2208.08933v1
• Date: Thu, 18 Aug 2022 16:09:12 GMT
• Title: Sequence Prediction Under Missing Data : An RNN Approach Without Imputation
• Authors: Soumen Pachal, Avinash Achar
• Abstract summary: This paper pertains to a novel Recurrent Network (RNN) based solution for sequence prediction under missing data. It tries to encode the missingness patterns in the data directly without trying to impute data either before or during model building. We focus on forecasting here in a general context of multi-step prediction in presence of possible inputs.
• Score: 1.9188864062289432
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Missing data scenarios are very common in ML applications in general and time-series/sequence applications are no exceptions. This paper pertains to a novel Recurrent Neural Network (RNN) based solution for sequence prediction under missing data. Our method is distinct from all existing approaches. It tries to encode the missingness patterns in the data directly without trying to impute data either before or during model building. Our encoding is lossless and achieves compression. It can be employed for both sequence classification and forecasting. We focus on forecasting here in a general context of multi-step prediction in presence of possible exogenous inputs. In particular, we propose novel variants of Encoder-Decoder (Seq2Seq) RNNs for this. The encoder here adopts the above mentioned pattern encoding, while at the decoder which has a different structure, multiple variants are feasible. We demonstrate the utility of our proposed architecture via multiple experiments on both single and multiple sequence (real) data-sets. We consider both scenarios where (i)data is naturally missing and (ii)data is synthetically masked.

Related papers

• Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth [83.15263499262824]
  We prove that gradient descent converges to a solution that completely disregards the sparse structure of the input. We show how to improve upon Gaussian performance for the compression of sparse data by adding a denoising function to a shallow architecture. We validate our findings on image datasets, such as CIFAR-10 and MNIST.
  arXiv  Detail & Related papers  (2024-02-07T16:32:29Z)
• Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting [0.0]
  Time series forecasting is a fundamental challenge in data science. We develop a parallel deep learning framework composed of RNN and CNN. Extensive experiments on three datasets reveal the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-05-08T17:20:13Z)
• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• HyperTime: Implicit Neural Representation for Time Series [131.57172578210256]
  Implicit neural representations (INRs) have recently emerged as a powerful tool that provides an accurate and resolution-independent encoding of data. In this paper, we analyze the representation of time series using INRs, comparing different activation functions in terms of reconstruction accuracy and training convergence speed. We propose a hypernetwork architecture that leverages INRs to learn a compressed latent representation of an entire time series dataset.
  arXiv  Detail & Related papers  (2022-08-11T14:05:51Z)
• Seasonal Encoder-Decoder Architecture for Forecasting [1.9188864062289432]
  We propose a novel RNN architecture capturing (stochastic) seasonal correlations intelligently. It is motivated from the well-known encoder-decoder (ED) architecture and multiplicative seasonal auto-regressive model. It can be employed on single or multiple sequence data.
  arXiv  Detail & Related papers  (2022-07-08T20:06:45Z)
• DenseHybrid: Hybrid Anomaly Detection for Dense Open-set Recognition [1.278093617645299]
  Anomaly detection can be conceived either through generative modelling of regular training data or by discriminating with respect to negative training data. This paper presents a novel hybrid anomaly score which allows dense open-set recognition on large natural images. Experiments evaluate our contributions on standard dense anomaly detection benchmarks as well as in terms of open-mIoU - a novel metric for dense open-set performance.
  arXiv  Detail & Related papers  (2022-07-06T11:48:50Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)
• Generalizing Variational Autoencoders with Hierarchical Empirical Bayes [6.273154057349038]
  We present Hierarchical Empirical Bayes Autoencoder (HEBAE), a computationally stable framework for probabilistic generative models. Our key contributions are two-fold. First, we make gains by placing a hierarchical prior over the encoding distribution, enabling us to adaptively balance the trade-off between minimizing the reconstruction loss function and avoiding over-regularization.
  arXiv  Detail & Related papers  (2020-07-20T18:18:39Z)
• Learning to Encode Position for Transformer with Continuous Dynamical Model [88.69870971415591]
  We introduce a new way of learning to encode position information for non-recurrent models, such as Transformer models. We model the evolution of encoded results along position index by such a dynamical system.
  arXiv  Detail & Related papers  (2020-03-13T00:41:41Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.