Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

• URL: http://arxiv.org/abs/2202.04599v1
• Date: Wed, 9 Feb 2022 17:50:52 GMT
• Title: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo
• Authors: Ignacio Peis, Chao Ma and Jos\'e Miguel Hern\'andez-Lobato
• Abstract summary: We present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data. Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation, supervised learning and outlier identification. We also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM.
• Score: 2.666288135543677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data and identifying outliers. However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations. To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference. Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation, supervised learning and outlier identification with missing features. Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM. Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.

Related papers

• Improved Anomaly Detection through Conditional Latent Space VAE Ensembles [49.1574468325115]
  Conditional Latent space Variational Autoencoder (CL-VAE) improved pre-processing for anomaly detection on data with known inlier classes and unknown outlier classes. Model shows increased accuracy in anomaly detection, achieving an AUC of 97.4% on the MNIST dataset. In addition, the CL-VAE shows increased benefits from ensembling, a more interpretable latent space, and an increased ability to learn patterns in complex data with limited model sizes.
  arXiv  Detail & Related papers  (2024-10-16T07:48:53Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Max-affine regression via first-order methods [7.12511675782289]
  The max-affine model ubiquitously arises in applications in signal processing and statistics. We present a non-asymptotic convergence analysis of gradient descent (GD) and mini-batch gradient descent (SGD) for max-affine regression.
  arXiv  Detail & Related papers  (2023-08-15T23:46:44Z)
• Consensus-Adaptive RANSAC [104.87576373187426]
  We propose a new RANSAC framework that learns to explore the parameter space by considering the residuals seen so far via a novel attention layer. The attention mechanism operates on a batch of point-to-model residuals, and updates a per-point estimation state to take into account the consensus found through a lightweight one-step transformer.
  arXiv  Detail & Related papers  (2023-07-26T08:25:46Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Learning Conditional Variational Autoencoders with Missing Covariates [0.8563354084119061]
  Conditional variational autoencoders (CVAEs) are versatile deep generative models. We develop computationally efficient methods to learn CVAEs and GP prior VAEs. Our experiments on simulated datasets as well as on a clinical trial study show that the proposed method outperforms previous methods.
  arXiv  Detail & Related papers  (2022-03-02T16:22:09Z)
• Learning Nonparametric Volterra Kernels with Gaussian Processes [0.0]
  This paper introduces a method for the nonparametric Bayesian learning of nonlinear operators, through the use of the Volterra series with kernels represented using Gaussian processes (GPs) When the input function to the operator is unobserved and has a GP prior, the NVKM constitutes a powerful method for both single and multiple output regression, and can be viewed as a nonlinear and nonparametric latent force model.
  arXiv  Detail & Related papers  (2021-06-10T08:21:00Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Model-based clustering of partial records [11.193504036335503]
  We develop clustering methodology through a model-based approach using the marginal density for the observed values. We compare our algorithm to the corresponding full expectation-maximization (EM) approach that considers the missing values in the incomplete data set. Simulation studies demonstrate that our approach has favorable recovery of the true cluster partition compared to case deletion and imputation.
  arXiv  Detail & Related papers  (2021-03-30T13:30:59Z)
• A Bayesian Approach with Type-2 Student-tMembership Function for T-S Model Identification [47.25472624305589]
  fuzzyc-regression clustering based on type-2 fuzzyset has been shown the remarkable results on non-sparse data. Aninnovative architecture for fuzzyc-regression model is presented and a novel student-tdistribution based membership functionis designed for sparse data modelling.
  arXiv  Detail & Related papers  (2020-09-02T05:10:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.