Disentangling representations in Restricted Boltzmann Machines without adversaries

• URL: http://arxiv.org/abs/2206.11600v1
• Date: Thu, 23 Jun 2022 10:24:20 GMT
• Title: Disentangling representations in Restricted Boltzmann Machines without adversaries
• Authors: Jorge Fernandez-de-Cossio-Diaz, Simona Cocco, Remi Monasson
• Abstract summary: We propose a simple, effective way of disentangling representations without any need to train adversarial discriminators. We show how our framework allows for computing the cost, in terms of log-likelihood of the data, associated to the disentanglement of their representations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A goal of unsupervised machine learning is to disentangle representations of complex high-dimensional data, allowing for interpreting the significant latent factors of variation in the data as well as for manipulating them to generate new data with desirable features. These methods often rely on an adversarial scheme, in which representations are tuned to avoid discriminators from being able to reconstruct specific data information (labels). We propose a simple, effective way of disentangling representations without any need to train adversarial discriminators, and apply our approach to Restricted Boltzmann Machines (RBM), one of the simplest representation-based generative models. Our approach relies on the introduction of adequate constraints on the weights during training, which allows us to concentrate information about labels on a small subset of latent variables. The effectiveness of the approach is illustrated on the MNIST dataset, the two-dimensional Ising model, and taxonomy of protein families. In addition, we show how our framework allows for computing the cost, in terms of log-likelihood of the data, associated to the disentanglement of their representations.

Related papers

• Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein [56.62376364594194]
  Unsupervised learning aims to capture the underlying structure of potentially large and high-dimensional datasets. In this work, we revisit these approaches under the lens of optimal transport and exhibit relationships with the Gromov-Wasserstein problem. This unveils a new general framework, called distributional reduction, that recovers DR and clustering as special cases and allows addressing them jointly within a single optimization problem.
  arXiv  Detail & Related papers  (2024-02-03T19:00:19Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• ProtoVAE: Prototypical Networks for Unsupervised Disentanglement [1.6114012813668934]
  We introduce a novel deep generative VAE-based model, ProtoVAE, that leverages a deep metric learning Prototypical network trained using self-supervision. Our model is completely unsupervised and requires no priori knowledge of the dataset, including the number of factors. We evaluate our proposed model on the benchmark dSprites, 3DShapes, and MPI3D disentanglement datasets.
  arXiv  Detail & Related papers  (2023-05-16T01:29:26Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Fair mapping [0.0]
  We propose a novel pre-processing method based on the transformation of the distribution of protected groups onto a chosen target one. We leverage on the recent works of the Wasserstein GAN and AttGAN frameworks to achieve the optimal transport of data points. Our proposed approach, preserves the interpretability of data and can be used without defining exactly the sensitive groups.
  arXiv  Detail & Related papers  (2022-09-01T17:31:27Z)
• Stable and Compact Face Recognition via Unlabeled Data Driven Sparse Representation-Based Classification [39.398339531136344]
  An unlabeled data driven inverse projection pseudo-full-space representation-based classification model is proposed. The proposed model aims to mine the hidden semantic information and intrinsic structure information of all available data. Experiments on three public datasets show that the proposed LR-S-PFSRC model achieves stable results.
  arXiv  Detail & Related papers  (2021-11-04T13:19:38Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Supervised Visualization for Data Exploration [9.742277703732187]
  We describe a novel supervised visualization technique based on random forest proximities and diffusion-based dimensionality reduction. Our approach is robust to noise and parameter tuning, thus making it simple to use while producing reliable visualizations for data exploration.
  arXiv  Detail & Related papers  (2020-06-15T19:10:17Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables [3.8073142980733]
  This paper provides a flexible, probabilistic framework that accounts for physical structure and information both in the training objectives and in the surrogate model itself. We advocate a probabilistic model in which equalities that are available from the physics can be introduced as virtual observables and can provide additional information through the likelihood.
  arXiv  Detail & Related papers  (2020-06-02T17:14:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.