Simplex Autoencoders

• URL: http://arxiv.org/abs/2301.06489v1
• Date: Mon, 16 Jan 2023 15:57:03 GMT
• Title: Simplex Autoencoders
• Authors: Aymene Mohammed Bouayed and David Naccache
• Abstract summary: We propose a new approach that models the latent space of an Autoencoder as a simplex, allowing for a novel for determining the number of components in the mixture model. We evaluate our approaches on a synthetic dataset and demonstrate their performance on three benchmark datasets.
• Score: 1.3960152426268768
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Synthetic data generation is increasingly important due to privacy concerns. While Autoencoder-based approaches have been widely used for this purpose, sampling from their latent spaces can be challenging. Mixture models are currently the most efficient way to sample from these spaces. In this work, we propose a new approach that models the latent space of an Autoencoder as a simplex, allowing for a novel heuristic for determining the number of components in the mixture model. This heuristic is independent of the number of classes and produces comparable results. We also introduce a sampling method based on probability mass functions, taking advantage of the compactness of the latent space. We evaluate our approaches on a synthetic dataset and demonstrate their performance on three benchmark datasets: MNIST, CIFAR-10, and Celeba. Our approach achieves an image generation FID of 4.29, 13.55, and 11.90 on the MNIST, CIFAR-10, and Celeba datasets, respectively. The best AE FID results to date on those datasets are respectively 6.3, 85.3 and 35.6 we hence substantially improve those figures (the lower is the FID the better). However, AEs are not the best performing algorithms on the concerned datasets and all FID records are currently held by GANs. While we do not perform better than GANs on CIFAR and Celeba we do manage to squeeze-out a non-negligible improvement (of 0.21) over the current GAN-held record for the MNIST dataset.

Related papers

• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity [2.095923926387536]
  Quality-Diversity (QD) algorithms have been proven to make grasp sampling significantly more efficient. We extend QDG-6DoF, a QD framework for generating object-centric grasps, to scale up the production of synthetic grasping datasets.
  arXiv  Detail & Related papers  (2024-10-03T08:56:14Z)
• Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models [69.76066070227452]
  *Data Synthesis* is a promising way to train a small model with very little labeled data. We propose *Synthesis Step by Step* (**S3**), a data synthesis framework that shrinks this distribution gap. Our approach improves the performance of a small model by reducing the gap between the synthetic dataset and the real data.
  arXiv  Detail & Related papers  (2023-10-20T17:14:25Z)
• Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision [6.891556476231427]
  Deep neural networks enable real-time monitoring of in-vehicle drivers, facilitating the timely prediction of distractions, fatigue, and potential hazards. Recent research has exposed unreliable cross-dataset driver behavior recognition due to a limited number of data samples and background noise. We propose a Score-Softmax classifier, which reduces the model overconfidence by enhancing category independence.
  arXiv  Detail & Related papers  (2023-10-08T15:28:01Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• Synthetic data, real errors: how (not) to publish and use synthetic data [86.65594304109567]
  We show how the generative process affects the downstream ML task. We introduce Deep Generative Ensemble (DGE) to approximate the posterior distribution over the generative process model parameters.
  arXiv  Detail & Related papers  (2023-05-16T07:30:29Z)
• Knowledge-integrated AutoEncoder Model [0.0]
  We introduce a novel approach for developing AE models that can integrate external knowledge sources into the learning process. The proposed model is evaluated on three large-scale datasets from three different scientific fields.
  arXiv  Detail & Related papers  (2023-03-12T18:00:12Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)
• Infinite Recommendation Networks: A Data-Centric Approach [8.044430277912936]
  We leverage the Neural Tangent Kernel to train infinitely-wide neural networks to devise $infty$-AE: an autoencoder with infinitely-wide bottleneck layers. We also develop Distill-CF for synthesizing tiny, high-fidelity data summaries. We observe 96-105% of $infty$-AE's performance on the full dataset with as little as 0.1% of the original dataset size.
  arXiv  Detail & Related papers  (2022-06-03T00:34:13Z)
• Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Mining [26.04006507181558]
  We propose a novel KG encoder -- Dual Attention Matching Network (Dual-AMN) The experimental results on widely used public datasets indicate that our method achieves both high accuracy and high efficiency.
  arXiv  Detail & Related papers  (2021-03-29T09:35:07Z)
• Least Squares Regression with Markovian Data: Fundamental Limits and Algorithms [69.45237691598774]
  We study the problem of least squares linear regression where the data-points are dependent and are sampled from a Markov chain. We establish sharp information theoretic minimax lower bounds for this problem in terms of $tau_mathsfmix$. We propose an algorithm based on experience replay--a popular reinforcement learning technique--that achieves a significantly better error rate.
  arXiv  Detail & Related papers  (2020-06-16T04:26:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.