Regularizing Neural Networks with Meta-Learning Generative Models
- URL: http://arxiv.org/abs/2307.13899v2
- Date: Mon, 23 Oct 2023 13:58:25 GMT
- Title: Regularizing Neural Networks with Meta-Learning Generative Models
- Authors: Shin'ya Yamaguchi, Daiki Chijiwa, Sekitoshi Kanai, Atsutoshi Kumagai,
Hisashi Kashima
- Abstract summary: We present a novel strategy for generative data augmentation called meta generative regularization (MGR)
MGR utilizes synthetic samples in the regularization term for feature extractors instead of in the loss function, e.g., cross-entropy.
Experiments on six datasets showed that MGR is effective particularly when datasets are smaller and stably outperforms baselines.
- Score: 40.45689466486025
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper investigates methods for improving generative data augmentation
for deep learning. Generative data augmentation leverages the synthetic samples
produced by generative models as an additional dataset for classification with
small dataset settings. A key challenge of generative data augmentation is that
the synthetic data contain uninformative samples that degrade accuracy. This is
because the synthetic samples do not perfectly represent class categories in
real data and uniform sampling does not necessarily provide useful samples for
tasks. In this paper, we present a novel strategy for generative data
augmentation called meta generative regularization (MGR). To avoid the
degradation of generative data augmentation, MGR utilizes synthetic samples in
the regularization term for feature extractors instead of in the loss function,
e.g., cross-entropy. These synthetic samples are dynamically determined to
minimize the validation losses through meta-learning. We observed that MGR can
avoid the performance degradation of na\"ive generative data augmentation and
boost the baselines. Experiments on six datasets showed that MGR is effective
particularly when datasets are smaller and stably outperforms baselines.
Related papers
- Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable.
We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data.
Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
arXiv Detail & Related papers (2024-10-29T04:14:32Z) - GANsemble for Small and Imbalanced Data Sets: A Baseline for Synthetic Microplastics Data [2.307414552248669]
This paper proposes GANsemble: a framework connecting data augmentation with conditional generative adversarial networks (cGANs) to generate class-conditioned synthetic data.
To our knowledge, this study is the first application of generative AI to synthetically create microplastics data.
arXiv Detail & Related papers (2024-04-10T21:23:13Z) - Synthetic data, real errors: how (not) to publish and use synthetic data [86.65594304109567]
We show how the generative process affects the downstream ML task.
We introduce Deep Generative Ensemble (DGE) to approximate the posterior distribution over the generative process model parameters.
arXiv Detail & Related papers (2023-05-16T07:30:29Z) - LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral
Image Generation with Variance Regularization [72.4394510913927]
Deep learning methods are state-of-the-art for spectral image (SI) computational tasks.
GANs enable diverse augmentation by learning and sampling from the data distribution.
GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation.
We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
arXiv Detail & Related papers (2023-04-29T00:25:02Z) - Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data.
We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
arXiv Detail & Related papers (2023-03-06T14:49:59Z) - DATGAN: Integrating expert knowledge into deep learning for synthetic
tabular data [0.0]
Synthetic data can be used in various applications, such as correcting bias datasets or replacing scarce original data for simulation purposes.
Deep learning models are data-driven and it is difficult to control the generation process.
This article presents the Directed Acyclic Tabular GAN ( DATGAN) to address these limitations.
arXiv Detail & Related papers (2022-03-07T16:09:03Z) - Negative Data Augmentation [127.28042046152954]
We show that negative data augmentation samples provide information on the support of the data distribution.
We introduce a new GAN training objective where we use NDA as an additional source of synthetic data for the discriminator.
Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities.
arXiv Detail & Related papers (2021-02-09T20:28:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.