S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation

• URL: http://arxiv.org/abs/2005.11437v1
• Date: Sat, 23 May 2020 00:44:38 GMT
• Title: S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
• Authors: Yizhe Zhu, Martin Renqiang Min, Asim Kadav, Hans Peter Graf
• Abstract summary: We propose a sequential variational autoencoder to learn disentangled representations of sequential data under self-supervision. We exploit the benefits of some readily accessible supervisory signals from input data itself or some off-the-shelf functional models. Our model can easily disentangle the representation of an input sequence into static factors and dynamic factors.
• Score: 31.38329747789168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a sequential variational autoencoder to learn disentangled representations of sequential data (e.g., videos and audios) under self-supervision. Specifically, we exploit the benefits of some readily accessible supervisory signals from input data itself or some off-the-shelf functional models and accordingly design auxiliary tasks for our model to utilize these signals. With the supervision of the signals, our model can easily disentangle the representation of an input sequence into static factors and dynamic factors (i.e., time-invariant and time-varying parts). Comprehensive experiments across videos and audios verify the effectiveness of our model on representation disentanglement and generation of sequential data, and demonstrate that, our model with self-supervision performs comparable to, if not better than, the fully-supervised model with ground truth labels, and outperforms state-of-the-art unsupervised models by a large margin.

Related papers

• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-Image Encoders [56.47577824219207]
  In this paper, we unveil the hidden costs associated with intrusive fine-tuning techniques. We introduce a new model reprogramming approach for fine-tuning, which we name Reprogrammer. Our empirical evidence reveals that Reprogrammer is less intrusive and yields superior downstream models.
  arXiv  Detail & Related papers  (2024-03-16T04:19:48Z)
• Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance [9.486558126032639]
  We introduce SyntaGuid, a novel approach to guide Transformer-based models towards critical source code tokens. We show that SyntaGuid can improve overall performance up to 3.25% and fix up to 28.3% wrong predictions.
  arXiv  Detail & Related papers  (2024-02-26T18:03:50Z)
• Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
  We develop an efficient, autoregression-based vision model on a limited dataset. We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding. Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
  arXiv  Detail & Related papers  (2024-02-07T13:41:53Z)
• A monitoring framework for deployed machine learning models with supply chain examples [2.904613270228912]
  We describe a framework for monitoring machine learning models; and, (2) its implementation for a big data supply chain application. We use our implementation to study drift in model features, predictions, and performance on three real data sets.
  arXiv  Detail & Related papers  (2022-11-11T14:31:38Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)
• Stacking VAE with Graph Neural Networks for Effective and Interpretable Time Series Anomaly Detection [5.935707085640394]
  We propose a stacking variational auto-encoder (VAE) model with graph neural networks for the effective and interpretable time-series anomaly detection. We show that our proposed model outperforms the strong baselines on three public datasets with considerable improvements.
  arXiv  Detail & Related papers  (2021-05-18T09:50:00Z)
• Generative Models as Distributions of Functions [72.2682083758999]
  Generative models are typically trained on grid-like data such as images. In this paper, we abandon discretized grids and instead parameterize individual data points by continuous functions.
  arXiv  Detail & Related papers  (2021-02-09T11:47:55Z)
• Disentangled Recurrent Wasserstein Autoencoder [17.769077848342334]
  recurrent Wasserstein Autoencoder (R-WAE) is a new framework for generative modeling of sequential data. R-WAE disentangles the representation of an input sequence into static and dynamic factors. Our models outperform other baselines with the same settings in terms of disentanglement and unconditional video generation.
  arXiv  Detail & Related papers  (2021-01-19T07:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.