Causally Disentangled Generative Variational AutoEncoder

• URL: http://arxiv.org/abs/2302.11737v2
• Date: Mon, 9 Oct 2023 01:32:17 GMT
• Title: Causally Disentangled Generative Variational AutoEncoder
• Authors: Seunghwan An, Kyungwoo Song, Jong-June Jeon
• Abstract summary: We present a new supervised learning technique for the Variational AutoEncoder (VAE) This technique allows it to learn a causally disentangled representation and generate causally disentangled outcomes simultaneously. We call this approach Causally Disentangled Generation (CDG)
• Score: 16.82544099843568
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present a new supervised learning technique for the Variational AutoEncoder (VAE) that allows it to learn a causally disentangled representation and generate causally disentangled outcomes simultaneously. We call this approach Causally Disentangled Generation (CDG). CDG is a generative model that accurately decodes an output based on a causally disentangled representation. Our research demonstrates that adding supervised regularization to the encoder alone is insufficient for achieving a generative model with CDG, even for a simple task. Therefore, we explore the necessary and sufficient conditions for achieving CDG within a specific model. Additionally, we introduce a universal metric for evaluating the causal disentanglement of a generative model. Empirical results from both image and tabular datasets support our findings.

Related papers

• Causal Diffusion Autoencoders: Toward Counterfactual Generation via Diffusion Probabilistic Models [17.124075103464392]
  Diffusion models (DPMs) have become the state-of-the-art in high-quality image generation. DPMs have an arbitrary noisy latent space with no interpretable or controllable semantics. We propose CausalDiffAE, a diffusion-based causal representation learning framework to enable counterfactual generation.
  arXiv  Detail & Related papers  (2024-04-27T00:09:26Z)
• Complexity Matters: Rethinking the Latent Space for Generative Modeling [65.64763873078114]
  In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion. In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity.
  arXiv  Detail & Related papers  (2023-07-17T07:12:29Z)
• Modeling Causal Mechanisms with Diffusion Models for Interventional and Counterfactual Queries [10.818661865303518]
  We consider the problem of answering observational, interventional, and counterfactual queries in a causally sufficient setting. We introduce diffusion-based causal models (DCM) to learn causal mechanisms, that generate unique latent encodings. Our empirical evaluations demonstrate significant improvements over existing state-of-the-art methods for answering causal queries.
  arXiv  Detail & Related papers  (2023-02-02T04:08:08Z)
• on the effectiveness of generative adversarial network on anomaly detection [1.6244541005112747]
  GANs rely on the rich contextual information of these models to identify the actual training distribution. We suggest a new unsupervised model based on GANs --a combination of an autoencoder and a GAN. A new scoring function was introduced to target anomalies where a linear combination of the internal representation of the discriminator and the generator's visual representation, plus the encoded representation of the autoencoder, come together to define the proposed anomaly score.
  arXiv  Detail & Related papers  (2021-12-31T16:35:47Z)
• P-WAE: Generalized Patch-Wasserstein Autoencoder for Anomaly Screening [17.24628770042803]
  We propose a novel Patch-wise Wasserstein AutoEncoder (P-WAE) architecture to alleviate those challenges. In particular, a patch-wise variational inference model coupled with solving the jigsaw puzzle is designed. Comprehensive experiments, conducted on the MVTec AD dataset, demonstrate the superior performance of our propo
  arXiv  Detail & Related papers  (2021-08-09T05:31:45Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)
• Unsupervised Controllable Generation with Self-Training [90.04287577605723]
  controllable generation with GANs remains a challenging research problem. We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-17T21:50:35Z)
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator [85.68825725223873]
  Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data. We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator. We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
  arXiv  Detail & Related papers  (2020-04-05T01:50:16Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.