Restrained Generative Adversarial Network against Overfitting in Numeric Data Augmentation

• URL: http://arxiv.org/abs/2010.13549v1
• Date: Mon, 26 Oct 2020 13:01:24 GMT
• Title: Restrained Generative Adversarial Network against Overfitting in Numeric Data Augmentation
• Authors: Wei Wang, Yimeng Chai, Tao Cui, Chuang Wang, Baohua Zhang, Yue Li, Yi An
• Abstract summary: Generative Adversarial Network (GAN) is one of the popular schemes to augment the image dataset. In our study we find the generator G in the GAN fails to generate numerical data in lower-dimensional spaces. We propose a theoretical restraint, independence on the loss function, to suppress the overfitting.
• Score: 9.265768052866786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent studies, Generative Adversarial Network (GAN) is one of the popular schemes to augment the image dataset. However, in our study we find the generator G in the GAN fails to generate numerical data in lower-dimensional spaces, and we address overfitting in the generation. By analyzing the Directed Graphical Model (DGM), we propose a theoretical restraint, independence on the loss function, to suppress the overfitting. Practically, as the Statically Restrained GAN (SRGAN) and Dynamically Restrained GAN (DRGAN), two frameworks are proposed to employ the theoretical restraint to the network structure. In the static structure, we predefined a pair of particular network topologies of G and D as the restraint, and quantify such restraint by the interpretable metric Similarity of the Restraint (SR). While for DRGAN we design an adjustable dropout module for the restraint function. In the widely carried out 20 group experiments, on four public numerical class imbalance datasets and five classifiers, the static and dynamic methods together produce the best augmentation results of 19 from 20; and both two methods simultaneously generate 14 of 20 groups of the top-2 best, proving the effectiveness and feasibility of the theoretical restraints.

Related papers

• Nested Annealed Training Scheme for Generative Adversarial Networks [54.70743279423088]
  This paper focuses on a rigorous mathematical theoretical framework: the composite-functional-gradient GAN (CFG) We reveal the theoretical connection between the CFG model and score-based models. We find that the training objective of the CFG discriminator is equivalent to finding an optimal D(x)
  arXiv  Detail & Related papers  (2025-01-20T07:44:09Z)
• Graph Structure Refinement with Energy-based Contrastive Learning [56.957793274727514]
  We introduce an unsupervised method based on a joint of generative training and discriminative training to learn graph structure and representation. We propose an Energy-based Contrastive Learning (ECL) guided Graph Structure Refinement (GSR) framework, denoted as ECL-GSR. ECL-GSR achieves faster training with fewer samples and memories against the leading baseline, highlighting its simplicity and efficiency in downstream tasks.
  arXiv  Detail & Related papers  (2024-12-20T04:05:09Z)
• On the Convergence of (Stochastic) Gradient Descent for Kolmogorov--Arnold Networks [56.78271181959529]
  Kolmogorov--Arnold Networks (KANs) have gained significant attention in the deep learning community. Empirical investigations demonstrate that KANs optimized via gradient descent (SGD) are capable of achieving near-zero training loss.
  arXiv  Detail & Related papers  (2024-10-10T15:34:10Z)
• A Non-negative VAE:the Generalized Gamma Belief Network [49.970917207211556]
  The gamma belief network (GBN) has demonstrated its potential for uncovering multi-layer interpretable latent representations in text data. We introduce the generalized gamma belief network (Generalized GBN) in this paper, which extends the original linear generative model to a more expressive non-linear generative model. We also propose an upward-downward Weibull inference network to approximate the posterior distribution of the latent variables.
  arXiv  Detail & Related papers  (2024-08-06T18:18:37Z)
• Generative adversarial learning with optimal input dimension and its adaptive generator architecture [3.9043107077488797]
  We introduce a novel framework called generalized GANs (G-GANs) By incorporating the group penalty and the architecture penalty, G-GANs have several intriguing features. Extensive experiments conducted with simulated and benchmark data demonstrate the superior performance of G-GANs.
  arXiv  Detail & Related papers  (2024-05-06T03:30:02Z)
• Self-Contrastive Graph Diffusion Network [1.14219428942199]
  We propose a novel framework called the Self-Contrastive Graph Diffusion Network (SCGDN) Our framework consists of two main components: the Attentional Module (AttM) and the Diffusion Module (DiFM) Unlike existing methodologies, SCGDN is an augmentation-free approach that avoids "sampling bias" and semantic drift.
  arXiv  Detail & Related papers  (2023-07-27T04:00:23Z)
• Generative adversarial networks for data-scarce spectral applications [0.0]
  We report on an application of GANs in the domain of synthetic spectral data generation. We show that CWGANs can act as a surrogate model with improved performance in the low-data regime.
  arXiv  Detail & Related papers  (2023-07-14T16:27:24Z)
• DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets [81.75973217676986]
  Gene regulatory networks (GRN) describe interactions between genes and their products that control gene expression and cellular function. Existing methods either focus on challenge (1), identifying cyclic structure from dynamics, or on challenge (2) learning complex Bayesian posteriors over DAGs, but not both. In this paper we leverage the fact that it is possible to estimate the "velocity" of gene expression with RNA velocity techniques to develop an approach that addresses both challenges.
  arXiv  Detail & Related papers  (2023-02-08T16:36:40Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
  This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation. In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN. Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
  arXiv  Detail & Related papers  (2021-12-15T09:50:25Z)
• Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data [125.7135706352493]
  Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator.
  arXiv  Detail & Related papers  (2021-11-12T18:13:45Z)
• Fuzzy Generative Adversarial Networks [0.0]
  Generative Adversarial Networks (GANs) are well-known tools for data generation and semi-supervised classification. This paper introduces techniques that show improvement in the GANs' regression capability through mean absolute error (MAE) and mean squared error (MSE) We show that adding a fuzzy logic layer can enhance GAN's ability to perform regression; the most desirable injection location is problem-specific.
  arXiv  Detail & Related papers  (2021-10-27T17:05:06Z)
• Deep Learning of Dynamic Subsurface Flow via Theory-guided Generative Adversarial Network [0.0]
  Theory-guided generative adversarial network (TgGAN) is proposed to solve dynamic partial differential equations (PDEs) TgGAN is proposed for dynamic subsurface flow with heterogeneous model parameters. Numerical results demonstrate that the TgGAN model is robust and reliable for deep learning of dynamic PDEs.
  arXiv  Detail & Related papers  (2020-06-02T02:53:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.