Generating Symbolic Reasoning Problems with Transformer GANs

• URL: http://arxiv.org/abs/2110.10054v3
• Date: Fri, 5 May 2023 09:13:00 GMT
• Title: Generating Symbolic Reasoning Problems with Transformer GANs
• Authors: Jens U. Kreber and Christopher Hahn
• Abstract summary: We study the capabilities of GANs and Wasserstein GANs equipped with Transformer encoders to generate sensible and challenging training data for symbolic reasoning domains.
• Score: 3.04585143845864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the capabilities of GANs and Wasserstein GANs equipped with Transformer encoders to generate sensible and challenging training data for symbolic reasoning domains. We conduct experiments on two problem domains where Transformers have been successfully applied recently: symbolic mathematics and temporal specifications in verification. Even without autoregression, our GAN models produce syntactically correct instances. We show that the generated data can be used as a substitute for real training data when training a classifier, and, especially, that training data can be generated from a dataset that is too small to be trained on directly. Using a GAN setting also allows us to alter the target distribution: We show that by adding a classifier uncertainty part to the generator objective, we obtain a dataset that is even harder to solve for a temporal logic classifier than our original dataset.

Related papers

• ReFormer: Generating Radio Fakes for Data Augmentation [0.49109372384514843]
  ReFormer is a generative AI (GAI) model that can efficiently generate synthetic radio-frequency (RF) data. We show how different transformer architectures and other design choices affect the quality of generated RF fakes.
  arXiv  Detail & Related papers  (2024-12-31T05:28:35Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• SMaRt: Improving GANs with Score Matching Regularity [94.81046452865583]
  Generative adversarial networks (GANs) usually struggle in learning from highly diverse data, whose underlying manifold is complex. We show that score matching serves as a promising solution to this issue thanks to its capability of persistently pushing the generated data points towards the real data manifold. We propose to improve the optimization of GANs with score matching regularity (SMaRt)
  arXiv  Detail & Related papers  (2023-11-30T03:05:14Z)
• TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network [4.989480853499916]
  Time-series data is one of the most common types of data used in medical machine learning applications. We introduce TTS-GAN, a transformer-based GAN which can successfully generate realistic synthetic time-series data sequences. We use visualizations and dimensionality reduction techniques to demonstrate the similarity of real and generated time-series data.
  arXiv  Detail & Related papers  (2022-02-06T03:05:47Z)
• Improving the quality of generative models through Smirnov transformation [1.3492000366723798]
  We propose a novel activation function to be used as output of the generator agent. It is based on the Smirnov probabilistic transformation and it is specifically designed to improve the quality of the generated data.
  arXiv  Detail & Related papers  (2021-10-29T17:01:06Z)
• Discriminative and Generative Transformer-based Models For Situation Entity Classification [8.029049649310211]
  We re-examine the situation entity (SE) classification task with varying amounts of available training data. We exploit a Transformer-based variational autoencoder to encode sentences into a lower dimensional latent space.
  arXiv  Detail & Related papers  (2021-09-15T17:07:07Z)
• Domain Transformer: Predicting Samples of Unseen, Future Domains [1.7310589008573272]
  We learn a domain transformer in an unsupervised manner that allows generating data of unseen domains. Our approach first matches independently learned latent representations of two given domains obtained from an auto-encoder using a Cycle-GAN. In turn, a transformation of the original samples can be learned that can be applied iteratively to extrapolate to unseen domains.
  arXiv  Detail & Related papers  (2021-06-10T21:20:00Z)
• Category-Learning with Context-Augmented Autoencoder [63.05016513788047]
  Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. We propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network.
  arXiv  Detail & Related papers  (2020-10-10T14:04:44Z)
• Unsupervised Controllable Generation with Self-Training [90.04287577605723]
  controllable generation with GANs remains a challenging research problem. We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-17T21:50:35Z)
• On Leveraging Pretrained GANs for Generation with Limited Data [83.32972353800633]
  generative adversarial networks (GANs) can generate highly realistic images, that are often indistinguishable (by humans) from real images. Most images so generated are not contained in a training dataset, suggesting potential for augmenting training sets with GAN-generated data. We leverage existing GAN models pretrained on large-scale datasets to introduce additional knowledge, following the concept of transfer learning. An extensive set of experiments is presented to demonstrate the effectiveness of the proposed techniques on generation with limited data.
  arXiv  Detail & Related papers  (2020-02-26T21:53:36Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.