Mixture Proportion Estimation Beyond Irreducibility
- URL: http://arxiv.org/abs/2306.01253v1
- Date: Fri, 2 Jun 2023 03:32:44 GMT
- Title: Mixture Proportion Estimation Beyond Irreducibility
- Authors: Yilun Zhu, Aaron Fjeldsted, Darren Holland, George Landon, Azaree
Lintereur, and Clayton Scott
- Abstract summary: We present a resampling-based meta-algorithm that takes any existing MPE algorithm designed to work under irreducibility and adapts it to work under our more general condition.
Our approach exhibits improved estimation performance relative to baseline methods and to a recently proposed regrouping-based algorithm.
- Score: 9.99177526976127
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The task of mixture proportion estimation (MPE) is to estimate the weight of
a component distribution in a mixture, given observations from both the
component and mixture. Previous work on MPE adopts the irreducibility
assumption, which ensures identifiablity of the mixture proportion. In this
paper, we propose a more general sufficient condition that accommodates several
settings of interest where irreducibility does not hold. We further present a
resampling-based meta-algorithm that takes any existing MPE algorithm designed
to work under irreducibility and adapts it to work under our more general
condition. Our approach empirically exhibits improved estimation performance
relative to baseline methods and to a recently proposed regrouping-based
algorithm.
Related papers
- Robust Mixture Learning when Outliers Overwhelm Small Groups [51.49063715477438]
We study the problem of estimating the means of well-separated mixtures when an adversary may add arbitrary outliers.
We propose an algorithm that obtains order-optimal error guarantees for each mixture mean with a minimal list-size overhead.
arXiv Detail & Related papers (2024-07-22T16:51:05Z) - Estimating Joint Probability Distribution With Low-Rank Tensor
Decomposition, Radon Transforms and Dictionaries [3.0892724364965005]
We describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components.
We combine two key ideas: dictionaries to represent 1-D densities, and random projections to estimate the joint distribution from 1-D marginals.
Our algorithm benefits from improved sample complexity over the previous dictionary-based approach by using 1-D marginals for reconstruction.
arXiv Detail & Related papers (2023-04-18T05:37:15Z) - Reweighted Mixup for Subpopulation Shift [63.1315456651771]
Subpopulation shift exists in many real-world applications, which refers to the training and test distributions that contain the same subpopulation groups but with different subpopulation proportions.
Importance reweighting is a classical and effective way to handle the subpopulation shift.
We propose a simple yet practical framework, called reweighted mixup, to mitigate the overfitting issue.
arXiv Detail & Related papers (2023-04-09T03:44:50Z) - Deep Anti-Regularized Ensembles provide reliable out-of-distribution
uncertainty quantification [4.750521042508541]
Deep ensemble often return overconfident estimates outside the training domain.
We show that an ensemble of networks with large weights fitting the training data are likely to meet these two objectives.
We derive a theoretical framework for this approach and show that the proposed optimization can be seen as a "water-filling" problem.
arXiv Detail & Related papers (2023-04-08T15:25:12Z) - RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and
Out Distribution Robustness [94.69774317059122]
We show that the effectiveness of the well celebrated Mixup can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss.
This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup.
arXiv Detail & Related papers (2022-06-29T09:44:33Z) - Recursive Inference for Variational Autoencoders [34.552283758419506]
Inference networks of traditional Variational Autoencoders (VAEs) are typically amortized.
Recent semi-amortized approaches were proposed to address this drawback.
We introduce an accurate amortized inference algorithm.
arXiv Detail & Related papers (2020-11-17T10:22:12Z) - Amortized Conditional Normalized Maximum Likelihood: Reliable Out of
Distribution Uncertainty Estimation [99.92568326314667]
We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation.
Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle.
We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
arXiv Detail & Related papers (2020-11-05T08:04:34Z) - Consistent Estimation of Identifiable Nonparametric Mixture Models from
Grouped Observations [84.81435917024983]
This work proposes an algorithm that consistently estimates any identifiable mixture model from grouped observations.
A practical implementation is provided for paired observations, and the approach is shown to outperform existing methods.
arXiv Detail & Related papers (2020-06-12T20:44:22Z) - Batch Stationary Distribution Estimation [98.18201132095066]
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions.
We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
arXiv Detail & Related papers (2020-03-02T09:10:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.