Robust Gaussian Process Regression with Huber Likelihood

• URL: http://arxiv.org/abs/2301.07858v1
• Date: Thu, 19 Jan 2023 02:59:33 GMT
• Title: Robust Gaussian Process Regression with Huber Likelihood
• Authors: Pooja Algikar and Lamine Mili
• Abstract summary: We propose a robust process model in the Gaussian process framework with the likelihood of observed data expressed as the Huber probability distribution. The proposed model employs weights based on projection statistics to scale residuals and bound the influence of vertical outliers and bad leverage points on the latent functions estimates.
• Score: 2.7184224088243365
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Gaussian process regression in its most simplified form assumes normal homoscedastic noise and utilizes analytically tractable mean and covariance functions of predictive posterior distribution using Gaussian conditioning. Its hyperparameters are estimated by maximizing the evidence, commonly known as type II maximum likelihood estimation. Unfortunately, Bayesian inference based on Gaussian likelihood is not robust to outliers, which are often present in the observational training data sets. To overcome this problem, we propose a robust process model in the Gaussian process framework with the likelihood of observed data expressed as the Huber probability distribution. The proposed model employs weights based on projection statistics to scale residuals and bound the influence of vertical outliers and bad leverage points on the latent functions estimates while exhibiting a high statistical efficiency at the Gaussian and thick tailed noise distributions. The proposed method is demonstrated by two real world problems and two numerical examples using datasets with additive errors following thick tailed distributions such as Students t, Laplace, and Cauchy distribution.

Related papers

• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• A Heavy-Tailed Algebra for Probabilistic Programming [53.32246823168763]
  We propose a systematic approach for analyzing the tails of random variables. We show how this approach can be used during the static analysis (before drawing samples) pass of a probabilistic programming language compiler. Our empirical results confirm that inference algorithms that leverage our heavy-tailed algebra attain superior performance across a number of density modeling and variational inference tasks.
  arXiv  Detail & Related papers  (2023-06-15T16:37:36Z)
• Bayesian approach to Gaussian process regression with uncertain inputs [0.0]
  In this work, we propose a Bayesian method that integrates the variability of input data into Gaussian process regression. The effectiveness of this new regression technique is demonstrated through several numerical examples.
  arXiv  Detail & Related papers  (2023-05-19T10:53:08Z)
• Maximum Likelihood Estimation in Gaussian Process Regression is Ill-Posed [7.018149356115115]
  It remains an open problem to establish the circumstances in which maximum likelihood estimation is well-posed. This article identifies scenarios where the maximum likelihood estimator fails to be well-posed. Although the failure of maximum likelihood estimation is part of Gaussian process folklore, these rigorous theoretical results appear to be the first of their kind.
  arXiv  Detail & Related papers  (2022-03-17T09:00:39Z)
• A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data [71.9573352891936]
  This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data.
  arXiv  Detail & Related papers  (2022-01-28T10:01:37Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• Latent Gaussian Model Boosting [0.0]
  Tree-boosting shows excellent predictive accuracy on many data sets. We obtain increased predictive accuracy compared to existing approaches in both simulated and real-world data experiments.
  arXiv  Detail & Related papers  (2021-05-19T07:36:30Z)
• Distributionally Robust Parametric Maximum Likelihood Estimation [13.09499764232737]
  We propose a distributionally robust maximum likelihood estimator that minimizes the worst-case expected log-loss uniformly over a parametric nominal distribution. Our novel robust estimator also enjoys statistical consistency and delivers promising empirical results in both regression and classification tasks.
  arXiv  Detail & Related papers  (2020-10-11T19:05:49Z)
• Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
  We propose an easy-to-use and general-purpose approach for fast posterior sampling. We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
  arXiv  Detail & Related papers  (2020-02-21T14:03:16Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)
• Robust Gaussian Process Regression with a Bias Model [0.6850683267295248]
  Most existing approaches replace an outlier-prone Gaussian likelihood with a non-Gaussian likelihood induced from a heavy tail distribution. The proposed approach models an outlier as a noisy and biased observation of an unknown regression function. Conditioned on the bias estimates, the robust GP regression can be reduced to a standard GP regression problem.
  arXiv  Detail & Related papers  (2020-01-14T06:21:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.