A Kernel Stein Test for Comparing Latent Variable Models

• URL: http://arxiv.org/abs/1907.00586v5
• Date: Tue, 9 May 2023 11:44:57 GMT
• Title: A Kernel Stein Test for Comparing Latent Variable Models
• Authors: Heishiro Kanagawa and Wittawat Jitkrittum and Lester Mackey and Kenji Fukumizu and Arthur Gretton
• Abstract summary: We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables. We show that our test significantly outperforms the relative Maximum Mean Discrepancy test, which is based on samples from the models and does not exploit the latent structure.
• Score: 48.32146056855925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable. The proposed test generalizes the recently proposed kernel Stein discrepancy (KSD) tests (Liu et al., 2016, Chwialkowski et al., 2016, Yang et al., 2018) to the case of latent variable models, a much more general class than the fully observed models treated previously. The new test, with a properly calibrated threshold, has a well-controlled type-I error. In the case of certain models with low-dimensional latent structure and high-dimensional observations, our test significantly outperforms the relative Maximum Mean Discrepancy test, which is based on samples from the models and does not exploit the latent structure.

Related papers

• On the Robustness of Kernel Goodness-of-Fit Tests [5.959410850280868]
  We show that existing kernel goodness-of-fit tests are not robust according to common notions of robustness. We propose the first robust kernel goodness-of-fit test which resolves this open problem using kernel Stein discrepancy balls.
  arXiv  Detail & Related papers  (2024-08-11T19:54:50Z)
• A kernel Stein test of goodness of fit for sequential models [19.8408003104988]
  The proposed measure is an instance of the kernel Stein discrepancy (KSD), which has been used to construct goodness-of-fit tests for unnormalized densities. We extend the KSD to the variable-dimension setting by identifying appropriate Stein operators, and propose a novel KSD goodness-of-fit test. Our test is shown to perform well in practice on discrete sequential data benchmarks.
  arXiv  Detail & Related papers  (2022-10-19T17:30:15Z)
• A Simple Unified Approach to Testing High-Dimensional Conditional Independences for Categorical and Ordinal Data [0.26651200086513094]
  Conditional independence (CI) tests underlie many approaches to model testing and structure learning in causal inference. Most existing CI tests for categorical and ordinal data stratify the sample by the conditioning variables, perform simple independence tests in each stratum, and combine the results. Here we propose a simple unified CI test for ordinal and categorical data that maintains reasonable calibration and power in high dimensions.
  arXiv  Detail & Related papers  (2022-06-09T08:56:12Z)
• Benign-Overfitting in Conditional Average Treatment Effect Prediction with Linear Regression [14.493176427999028]
  We study the benign overfitting theory in the prediction of the conditional average treatment effect (CATE) with linear regression models. We show that the T-learner fails to achieve the consistency except the random assignment, while the IPW-learner converges the risk to zero if the propensity score is known.
  arXiv  Detail & Related papers  (2022-02-10T18:51:52Z)
• Composite Goodness-of-fit Tests with Kernels [19.744607024807188]
  We propose a kernel-based hypothesis tests for the challenging composite testing problem. Our tests make use of minimum distance estimators based on the maximum mean discrepancy and the kernel Stein discrepancy. As our main result, we show that we are able to estimate the parameter and conduct our test on the same data, while maintaining a correct test level.
  arXiv  Detail & Related papers  (2021-11-19T15:25:06Z)
• Optimization Variance: Exploring Generalization Properties of DNNs [83.78477167211315]
  The test error of a deep neural network (DNN) often demonstrates double descent. We propose a novel metric, optimization variance (OV), to measure the diversity of model updates.
  arXiv  Detail & Related papers  (2021-06-03T09:34:17Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations [98.3066727301239]
  We identify two key properties of the training data that drive this behavior. We show how the inductive bias of models towards "memorizing" fewer examples can cause over parameterization to hurt.
  arXiv  Detail & Related papers  (2020-05-09T01:59:13Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)
• Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
  We show that a posterior approximation distinct from the variational distribution should be used for making decisions. Motivated by these theoretical results, we propose learning several approximate proposals for the best model. In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
  arXiv  Detail & Related papers  (2020-02-17T19:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.