Mixed Membership sub-Gaussian Models

• URL: http://arxiv.org/abs/2604.22633v1
• Date: Fri, 24 Apr 2026 15:07:32 GMT
• Title: Mixed Membership sub-Gaussian Models
• Abstract summary: We develop an efficient algorithm to estimate the mixed membership of each individual observation.<n>We prove that the estimation error of the per-individual membership vector can be made arbitrarily small with high probability.
• Score: 4.314956204483074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Gaussian mixture model is widely used in unsupervised learning, owing to its simplicity and interpretability. However, a fundamental limitation of the classical Gaussian mixture model is that it forces each observation to belong to exactly one component. In many practical applications, such as genetics, social network analysis, and text mining, an observation may naturally belong to multiple components or exhibit partial membership in several latent components. To overcome this limitation, we propose the mixed membership sub-Gaussian model, which extends the classical Gaussian mixture framework by allowing each observation to belong to multiple components. This model inherits the interpretability of the classical Gaussian mixture model while offering greater flexibility for capturing complex overlapping structures. We develop an efficient spectral algorithm to estimate the mixed membership of each individual observation, and under mild separation conditions on the component centres, we prove that the estimation error of the per-individual membership vector can be made arbitrarily small with high probability. To our knowledge, this is the first work to provide a computationally efficient estimator with such a vanishing-error guarantee for a mixed-membership extension of the Gaussian mixture model. Extensive experimental studies demonstrate that our method outperforms existing approaches that ignore mixed memberships.

Related papers

• Individual-heterogeneous sub-Gaussian Mixture Models [4.314956204483074]
  We introduce the individual-heterogeneous sub-Gaussian mixture model.<n>We propose an efficient spectral method that provably achieves exact recovery of the true cluster labels.<n> Numerical experiments on both synthetic and real data demonstrate that our method consistently outperforms existing clustering algorithms.
  arXiv  Detail & Related papers  (2026-04-07T02:22:51Z)
• A Bayesian approach to learning mixtures of nonparametric components [12.42953971157138]
  In many applications, making parametric assumptions on the latent subpopulation distributions may be unrealistic.<n>We study finite mixtures with nonparametric mixture components, using a Bayesian nonparametric modeling approach.
  arXiv  Detail & Related papers  (2025-12-15T05:27:01Z)
• Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles [12.889537265772242]
  We introduce a new family of parsimonious GMMs with piecewise-constant covariance eigenvalue profiles.<n>We show the superior likelihood-parsimony tradeoffs achieved by our models on a variety of unsupervised experiments.
  arXiv  Detail & Related papers  (2025-07-02T09:52:56Z)
• Model Selection for Gaussian-gated Gaussian Mixture of Experts Using Dendrograms of Mixing Measures [24.865197779389323]
  Mixture of Experts (MoE) models constitute a widely utilized class of ensemble learning approaches in statistics and machine learning.<n>We introduce a novel extension to Gaussian-gated MoE models that enables consistent estimation of the true number of mixture components.<n> Experimental results on synthetic data demonstrate the effectiveness of the proposed method in accurately recovering the number of experts.
  arXiv  Detail & Related papers  (2025-05-19T12:41:19Z)
• Adaptive Fuzzy C-Means with Graph Embedding [84.47075244116782]
  Fuzzy clustering algorithms can be roughly categorized into two main groups: Fuzzy C-Means (FCM) based methods and mixture model based methods. We propose a novel FCM based clustering model that is capable of automatically learning an appropriate membership degree hyper- parameter value.
  arXiv  Detail & Related papers  (2024-05-22T08:15:50Z)
• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data [71.9573352891936]
  This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data.
  arXiv  Detail & Related papers  (2022-01-28T10:01:37Z)
• Mixed membership distribution-free model [0.4972323953932129]
  We consider the problem of community detection in overlapping weighted networks, where nodes can belong to multiple communities and edge weights can be finite real numbers. To model such complex networks, we propose a general framework - the mixed membership distribution-free (MMDF) model. We use an efficient spectral algorithm with a theoretical guarantee of convergence rate to estimate community memberships under the model.
  arXiv  Detail & Related papers  (2021-12-04T18:21:02Z)
• Shared Independent Component Analysis for Multi-Subject Neuroimaging [107.29179765643042]
  We introduce Shared Independent Component Analysis (ShICA) that models each view as a linear transform of shared independent components contaminated by additive Gaussian noise. We show that this model is identifiable if the components are either non-Gaussian or have enough diversity in noise variances. We provide empirical evidence on fMRI and MEG datasets that ShICA yields more accurate estimation of the components than alternatives.
  arXiv  Detail & Related papers  (2021-10-26T08:54:41Z)
• Fitting large mixture models using stochastic component selection [0.0]
  We propose a combination of the expectation of the computational and the Metropolis-Hastings algorithm to evaluate only a small number of components. The Markov chain of component assignments is sequentially generated across the algorithm's iterations. We put emphasis on generality of our method, equipping it with the ability to train both shallow and deep mixture models.
  arXiv  Detail & Related papers  (2021-10-10T12:39:53Z)
• Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions [79.35722941720734]
  Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. We prove exacts characterising the estimator in high-dimensions via empirical risk minimisation. We discuss how our theory can be applied beyond the scope of synthetic data.
  arXiv  Detail & Related papers  (2021-06-07T16:53:56Z)
• Consistent Estimation of Identifiable Nonparametric Mixture Models from Grouped Observations [84.81435917024983]
  This work proposes an algorithm that consistently estimates any identifiable mixture model from grouped observations. A practical implementation is provided for paired observations, and the approach is shown to outperform existing methods.
  arXiv  Detail & Related papers  (2020-06-12T20:44:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.