Predictive Coresets

• URL: http://arxiv.org/abs/2502.05725v1
• Date: Sat, 08 Feb 2025 23:57:43 GMT
• Title: Predictive Coresets
• Authors: Bernardo Flores,
• Abstract summary: Conventional coreset approaches determine weights by minimizing the Kullback-Leibler divergence between the likelihood functions of the full and weighted datasets. We propose an alternative variational method which employs randomized posteriors and finds weights to match the unknown posterior predictive distributions conditioned on the full and reduced datasets. We evaluate the performance of the proposed coreset construction on diverse problems, including random partitions and density estimation.
• Score: 0.0
• License:
• Abstract: Modern data analysis often involves massive datasets with hundreds of thousands of observations, making traditional inference algorithms computationally prohibitive. Coresets are selection methods designed to choose a smaller subset of observations while maintaining similar learning performance. Conventional coreset approaches determine these weights by minimizing the Kullback-Leibler (KL) divergence between the likelihood functions of the full and weighted datasets; as a result, this makes them ill-posed for nonparametric models, where the likelihood is often intractable. We propose an alternative variational method which employs randomized posteriors and finds weights to match the unknown posterior predictive distributions conditioned on the full and reduced datasets. Our approach provides a general algorithm based on predictive recursions suitable for nonparametric priors. We evaluate the performance of the proposed coreset construction on diverse problems, including random partitions and density estimation.

Related papers

• Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance [52.093434664236014]
  Recent diffusion models provide a promising zero-shot solution to noisy linear inverse problems without retraining for specific inverse problems. Inspired by this finding, we propose to improve recent methods by using more principled covariance determined by maximum likelihood estimation.
  arXiv  Detail & Related papers  (2024-02-03T13:35:39Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Bayesian Hierarchical Models for Counterfactual Estimation [12.159830463756341]
  We propose a probabilistic paradigm to estimate a diverse set of counterfactuals. We treat the perturbations as random variables endowed with prior distribution functions. A gradient based sampler with superior convergence characteristics efficiently computes the posterior samples.
  arXiv  Detail & Related papers  (2023-01-21T00:21:11Z)
• Black-box Coreset Variational Inference [13.892427580424444]
  We present a black-box variational inference framework for coresets to enable principled application of variational coresets to intractable models. We apply our techniques to supervised learning problems, and compare them with existing approaches in the literature for data summarization and inference.
  arXiv  Detail & Related papers  (2022-11-04T11:12:09Z)
• Fast Estimation of Bayesian State Space Models Using Amortized Simulation-Based Inference [0.0]
  This paper presents a fast algorithm for estimating hidden states of Bayesian state space models. After pretraining, finding the posterior distribution for any dataset takes from hundredths to tenths of a second.
  arXiv  Detail & Related papers  (2022-10-13T16:37:05Z)
• A similarity-based Bayesian mixture-of-experts model [0.5156484100374058]
  We present a new non-parametric mixture-of-experts model for multivariate regression problems. Using a conditionally specified model, predictions for out-of-sample inputs are based on similarities to each observed data point. Posterior inference is performed on the parameters of the mixture as well as the distance metric.
  arXiv  Detail & Related papers  (2020-12-03T18:08:30Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• Sparse Cholesky covariance parametrization for recovering latent structure in ordered data [1.5349431582672617]
  We focus on arbitrary zero patterns in the Cholesky factor of a covariance matrix. For the ordered scenario, we propose a novel estimation method that is based on matrix loss penalization. We give guidelines, based on the empirical results, about which of the methods analysed is more appropriate for each setting.
  arXiv  Detail & Related papers  (2020-06-02T08:35:00Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.