Moment Matching Deep Contrastive Latent Variable Models

• URL: http://arxiv.org/abs/2202.10560v1
• Date: Mon, 21 Feb 2022 22:27:30 GMT
• Title: Moment Matching Deep Contrastive Latent Variable Models
• Authors: Ethan Weinberger, Nicasia Beebe-Wang, Su-In Lee
• Abstract summary: In contrastive analysis, machine learning practitioners are interested in discovering patterns that are enriched in a target dataset as compared to a background dataset. Here we propose the moment matching contrastive VAE (MM-cVAE), a reformulation of the VAE for CA. On three challenging CA tasks we find that our method outperforms the previous state-of-the-art both qualitatively and on a set of quantitative metrics.
• Score: 11.602089225841631
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the contrastive analysis (CA) setting, machine learning practitioners are specifically interested in discovering patterns that are enriched in a target dataset as compared to a background dataset generated from sources of variation irrelevant to the task at hand. For example, a biomedical data analyst may seek to understand variations in genomic data only present among patients with a given disease as opposed to those also present in healthy control subjects. Such scenarios have motivated the development of contrastive latent variable models to isolate variations unique to these target datasets from those shared across the target and background datasets, with current state of the art models based on the variational autoencoder (VAE) framework. However, previously proposed models do not explicitly enforce the constraints on latent variables underlying CA, potentially leading to the undesirable leakage of information between the two sets of latent variables. Here we propose the moment matching contrastive VAE (MM-cVAE), a reformulation of the VAE for CA that uses the maximum mean discrepancy to explicitly enforce two crucial latent variable constraints underlying CA. On three challenging CA tasks we find that our method outperforms the previous state-of-the-art both qualitatively and on a set of quantitative metrics.

Related papers

• Causal Effect Estimation using identifiable Variational AutoEncoder with Latent Confounders and Post-Treatment Variables [18.34462010115951]
  Estimating causal effects from observational data is challenging, especially in the presence of latent confounders. We propose a novel method of joint Variational AutoEncoder (VAE) and identifiable Variational AutoEncoder (iVAE) for learning the representations of latent confounders and latent post-treatment variables.
  arXiv  Detail & Related papers  (2024-08-13T22:13:25Z)
• SAVAE: Leveraging the variational Bayes autoencoder for survival analysis [10.0060346233449]
  We introduce SAVAE (Survival Analysis Variational Autoencoder), a novel approach based on Variational Autoencoders. Savoe contributes significantly to the field by introducing a tailored ELBO formulation for survival analysis. It offers a general method that consistently performs well on various metrics, demonstrating robustness and stability through different experiments.
  arXiv  Detail & Related papers  (2023-12-22T12:36:50Z)
• Model Stealing Attack against Graph Classification with Authenticity, Uncertainty and Diversity [80.16488817177182]
  GNNs are vulnerable to the model stealing attack, a nefarious endeavor geared towards duplicating the target model via query permissions. We introduce three model stealing attacks to adapt to different actual scenarios.
  arXiv  Detail & Related papers  (2023-12-18T05:42:31Z)
• SepVAE: a contrastive VAE to separate pathological patterns from healthy ones [2.619659560375341]
  Contrastive Analysis VAE (CA-VAEs) is a family of Variational auto-encoders (VAEs) that aims at separating the common factors of variation between a background dataset (BG) and a target dataset (TG) We show a better performance than previous CA-VAEs methods on three medical applications and a natural images dataset (CelebA)
  arXiv  Detail & Related papers  (2023-07-12T14:52:21Z)
• Causal Inference via Nonlinear Variable Decorrelation for Healthcare Applications [60.26261850082012]
  We introduce a novel method with a variable decorrelation regularizer to handle both linear and nonlinear confounding. We employ association rules as new representations using association rule mining based on the original features to increase model interpretability.
  arXiv  Detail & Related papers  (2022-09-29T17:44:14Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Selecting Treatment Effects Models for Domain Adaptation Using Causal Knowledge [82.5462771088607]
  We propose a novel model selection metric specifically designed for ITE methods under the unsupervised domain adaptation setting. In particular, we propose selecting models whose predictions of interventions' effects satisfy known causal structures in the target domain.
  arXiv  Detail & Related papers  (2021-02-11T21:03:14Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• NestedVAE: Isolating Common Factors via Weak Supervision [45.366986365879505]
  We identify the connection between the task of bias reduction and that of isolating factors common between domains. To isolate the common factors we combine the theory of deep latent variable models with information bottleneck theory. Two outer VAEs with shared weights attempt to reconstruct the input and infer a latent space, whilst a nested VAE attempts to reconstruct the latent representation of one image, from the latent representation of its paired image.
  arXiv  Detail & Related papers  (2020-02-26T15:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.