Generative adversarial learning with optimal input dimension and its adaptive generator architecture

• URL: http://arxiv.org/abs/2405.03723v1
• Date: Mon, 6 May 2024 03:30:02 GMT
• Title: Generative adversarial learning with optimal input dimension and its adaptive generator architecture
• Authors: Zhiyao Tan, Ling Zhou, Huazhen Lin,
• Abstract summary: We introduce a novel framework called generalized GANs (G-GANs) By incorporating the group penalty and the architecture penalty, G-GANs have several intriguing features. Extensive experiments conducted with simulated and benchmark data demonstrate the superior performance of G-GANs.
• Score: 3.9043107077488797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate the impact of the input dimension on the generalization error in generative adversarial networks (GANs). In particular, we first provide both theoretical and practical evidence to validate the existence of an optimal input dimension (OID) that minimizes the generalization error. Then, to identify the OID, we introduce a novel framework called generalized GANs (G-GANs), which includes existing GANs as a special case. By incorporating the group penalty and the architecture penalty developed in the paper, G-GANs have several intriguing features. First, our framework offers adaptive dimensionality reduction from the initial dimension to a dimension necessary for generating the target distribution. Second, this reduction in dimensionality also shrinks the required size of the generator network architecture, which is automatically identified by the proposed architecture penalty. Both reductions in dimensionality and the generator network significantly improve the stability and the accuracy of the estimation and prediction. Theoretical support for the consistent selection of the input dimension and the generator network is provided. Third, the proposed algorithm involves an end-to-end training process, and the algorithm allows for dynamic adjustments between the input dimension and the generator network during training, further enhancing the overall performance of G-GANs. Extensive experiments conducted with simulated and benchmark data demonstrate the superior performance of G-GANs. In particular, compared to that of off-the-shelf methods, G-GANs achieves an average improvement of 45.68% in the CT slice dataset, 43.22% in the MNIST dataset and 46.94% in the FashionMNIST dataset in terms of the maximum mean discrepancy or Frechet inception distance. Moreover, the features generated based on the input dimensions identified by G-GANs align with visually significant features.

Related papers

• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution. Our contribution is to provide the first general estimation technique for transportability problems. We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• A Genetic Algorithm-Based Approach for Automated Optimization of Kolmogorov-Arnold Networks in Classification Tasks [8.669319624657701]
  Kolmogorov-Arnold Networks (KANs) are introduced in 2024 to address the issue of interpretability in multilayer perceptrons. This paper proposes GA-KAN, a genetic-based approach that automates the optimization of KANs, requiring no human intervention in the design process. GA-KAN is validated on two toy datasets, achieving optimal results without the manual tuning required by the original KAN.
  arXiv  Detail & Related papers  (2025-01-29T04:32:36Z)
• Diffusion Models as Network Optimizers: Explorations and Analysis [71.69869025878856]
  generative diffusion models (GDMs) have emerged as a promising new approach to network optimization. In this study, we first explore the intrinsic characteristics of generative models. We provide a concise theoretical and intuitive demonstration of the advantages of generative models over discriminative network optimization.
  arXiv  Detail & Related papers  (2024-11-01T09:05:47Z)
• Predicting Transonic Flowfields in Non-Homogeneous Unstructured Grids Using Autoencoder Graph Convolutional Networks [0.0]
  This paper focuses on addressing challenges posed by non-homogeneous unstructured grids, commonly used in Computational Fluid Dynamics (CFD) The core of our approach centers on geometric deep learning, specifically the utilization of graph convolutional network (GCN) The novel Autoencoder GCN architecture enhances prediction accuracy by propagating information to distant nodes and emphasizing influential points.
  arXiv  Detail & Related papers  (2024-05-07T15:18:21Z)
• LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral Image Generation with Variance Regularization [72.4394510913927]
  Deep learning methods are state-of-the-art for spectral image (SI) computational tasks. GANs enable diverse augmentation by learning and sampling from the data distribution. GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation. We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
  arXiv  Detail & Related papers  (2023-04-29T00:25:02Z)
• Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization [4.0554893636822]
  We introduce a novel approach to deploy large-scale Deep Neural Networks on constrained resources. The method speeds up inference time and aims to reduce memory demand and power consumption.
  arXiv  Detail & Related papers  (2022-12-25T15:40:05Z)
• Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics [6.940097162264939]
  randomness makes SCNs more likely to generate approximate linear correlative nodes that are redundant and low quality. In light of a fundamental principle in machine learning, that is, a model with fewer parameters holds improved generalization. This paper proposes orthogonal SCN, termed OSCN, to filtrate out the low-quality hidden nodes for network structure reduction.
  arXiv  Detail & Related papers  (2022-05-26T07:07:26Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)
• Restrained Generative Adversarial Network against Overfitting in Numeric Data Augmentation [9.265768052866786]
  Generative Adversarial Network (GAN) is one of the popular schemes to augment the image dataset. In our study we find the generator G in the GAN fails to generate numerical data in lower-dimensional spaces. We propose a theoretical restraint, independence on the loss function, to suppress the overfitting.
  arXiv  Detail & Related papers  (2020-10-26T13:01:24Z)
• Improving Generative Adversarial Networks with Local Coordinate Coding [150.24880482480455]
  Generative adversarial networks (GANs) have shown remarkable success in generating realistic data from some predefined prior distribution. In practice, semantic information might be represented by some latent distribution learned from data. We propose an LCCGAN model with local coordinate coding (LCC) to improve the performance of generating data.
  arXiv  Detail & Related papers  (2020-07-28T09:17:50Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• Towards GANs' Approximation Ability [8.471366736328811]
  This paper will first theoretically analyze GANs' approximation property. We prove that the generator with the input latent variable in GANs can universally approximate the potential data distribution. In the practical dataset, four GANs using SDG can also outperform the corresponding traditional GANs when the model architectures are smaller.
  arXiv  Detail & Related papers  (2020-04-10T02:40:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.