Mixture Representation Learning with Coupled Autoencoders

• URL: http://arxiv.org/abs/2007.09880v3
• Date: Tue, 13 Apr 2021 02:02:27 GMT
• Title: Mixture Representation Learning with Coupled Autoencoders
• Abstract summary: We propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE. In this framework, the mixture representation of each network is regularized by imposing a consensus constraint on the discrete factor. We use the proposed method to jointly uncover discrete and continuous factors of variability describing gene expression in a single-cell transcriptomic dataset.
• Score: 1.589915930948668
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Jointly identifying a mixture of discrete and continuous factors of variability without supervision is a key problem in unraveling complex phenomena. Variational inference has emerged as a promising method to learn interpretable mixture representations. However, posterior approximation in high-dimensional latent spaces, particularly for discrete factors remains challenging. Here, we propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE that scales well to high-dimensional discrete settings. In this framework, the mixture representation of each network is regularized by imposing a consensus constraint on the discrete factor. We justify the use of this framework by providing both theoretical and experimental results. Finally, we use the proposed method to jointly uncover discrete and continuous factors of variability describing gene expression in a single-cell transcriptomic dataset profiling more than a hundred cortical neuron types.

Related papers

• Impute-EM: Native Mixed-State Diffusion Models for Heterogeneous Data Imputation [71.74203645336385]
  Impute-EM alternates between imputing missing entries with the current model and refitting a diffusion backbone on completed data.<n>We characterize the update and show that the observed mask-indexed marginals match the targets at the limit.
  arXiv  Detail & Related papers  (2026-09-14T09:39:13Z)
• Causal Variational Deep Embedding: A Family of Interventional Generators for Confounded Images [23.146237151956765]
  We show that CauVaDE produces diverse interventional samples and improves FID against an unconfounded reference.<n> Experiments on image data benchmarks show that CauVaDE produces diverse interventional samples and improves FID against an unconfounded reference.
  arXiv  Detail & Related papers  (2026-06-19T23:59:14Z)
• Implicit Variational Rejection Sampling [49.24060583051337]
  Variational Inference (VI) is a fundamental inference technique in Bayesian machine learning for approxing complex posterior distributions.<n>Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility.<n>We propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation.
  arXiv  Detail & Related papers  (2026-06-12T08:17:45Z)
• Disentanglement with Holographic Reduced Representations [36.48134953186487]
  Disentanglement, the separation of factors of variation in data using neural networks, remains a long-standing challenge in machine learning.<n>We introduce an unsupervised learning algorithm that uses holographic reduced representations (HRR) for neural disentanglement.<n>We show that the HRR unbinding operation provides an inductive bias for separating factors and yields competitive results against baselines.
  arXiv  Detail & Related papers  (2026-06-08T16:48:35Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• Learning Multi-type heterogeneous interacting particle systems [8.56664199108]
  We propose a framework for joint inference of network topology, multi-type interaction kernels, and type assignments in heterogeneous systems.<n>We provide theoretical guarantees with estimation bounds under the Isometry Property (RIP) assumption and establish conditions for the exact recovery interaction types based on separability.
  arXiv  Detail & Related papers  (2026-02-03T19:17:36Z)
• MissHDD: Hybrid Deterministic Diffusion for Hetrogeneous Incomplete Data Imputation [4.935498694293104]
  We propose a hybrid deterministic diffusion framework that separates heterogeneous features into two complementary generative channels.<n>A continuous DDIM-based channel provides efficient and stable deterministic denoising for numerical variables.<n>A discrete latent-path diffusion channel, inspired by loopholing-based discrete diffusion, models categorical and discrete features without leaving their valid sample.<n>The two channels are trained under a unified conditional imputation objective, enabling coherent reconstruction of mixed-type incomplete data.
  arXiv  Detail & Related papers  (2025-11-18T14:44:49Z)
• Unlasting: Unpaired Single-Cell Multi-Perturbation Estimation by Dual Conditional Diffusion Implicit Bridges [68.98973318553983]
  We propose a framework based on Dual Diffusion Implicit Bridges (DDIB) to learn the mapping between different data distributions.<n>We integrate gene regulatory network (GRN) information to propagate perturbation signals in a biologically meaningful way.<n>We also incorporate a masking mechanism to predict silent genes, improving the quality of generated profiles.
  arXiv  Detail & Related papers  (2025-06-26T09:05:38Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• A Generalized Multiscale Bundle-Based Hyperspectral Sparse Unmixing Algorithm [8.616208042031877]
  In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporating group sparsity-inducing mixed norms.
  arXiv  Detail & Related papers  (2024-01-24T00:37:14Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• Structure-preserving GANs [6.438897276587413]
  We introduce structure-preserving GANs as a data-efficient framework for learning distributions. We show that we can reduce the discriminator space to its projection on the invariant discriminator space. We contextualize our framework by building symmetry-preserving GANs for distributions with intrinsic group symmetry.
  arXiv  Detail & Related papers  (2022-02-02T16:40:04Z)
• Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension [25.711297863946193]
  We develop a theory for the study of fluctuations in an ensemble of generalised linear models trained on different, but correlated, features. We provide a complete description of the joint distribution of the empirical risk minimiser for generic convex loss and regularisation in the high-dimensional limit.
  arXiv  Detail & Related papers  (2022-01-31T17:44:58Z)
• Sparse Communication via Mixed Distributions [29.170302047339174]
  We build theoretical foundations for "mixed random variables" Our framework suggests two strategies for representing and sampling mixed random variables. We experiment with both approaches on an emergent communication benchmark.
  arXiv  Detail & Related papers  (2021-08-05T14:49:03Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.