An information theoretic limit to data amplification

• URL: http://arxiv.org/abs/2412.18041v1
• Date: Mon, 23 Dec 2024 23:27:51 GMT
• Title: An information theoretic limit to data amplification
• Authors: S. J. Watts, L. Crow,
• Abstract summary: Generative Adversarial Networks (GANs) have been trained using Monte Carlo simulated input and then used to generate data for the same problem.<n>N training events for a GAN can result in generated events with the gain factor, G, being more than one.<n>It is shown that a gain of greater than one is possible whilst keeping the information content of the data unchanged.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years generative artificial intelligence has been used to create data to support science analysis. For example, Generative Adversarial Networks (GANs) have been trained using Monte Carlo simulated input and then used to generate data for the same problem. This has the advantage that a GAN creates data in a significantly reduced computing time. N training events for a GAN can result in GN generated events with the gain factor, G, being more than one. This appears to violate the principle that one cannot get information for free. This is not the only way to amplify data so this process will be referred to as data amplification which is studied using information theoretic concepts. It is shown that a gain of greater than one is possible whilst keeping the information content of the data unchanged. This leads to a mathematical bound which only depends on the number of generated and training events. This study determines conditions on both the underlying and reconstructed probability distributions to ensure this bound. In particular, the resolution of variables in amplified data is not improved by the process but the increase in sample size can still improve statistical significance. The bound is confirmed using computer simulation and analysis of GAN generated data from the literature.

Related papers

• D3A-TS: Denoising-Driven Data Augmentation in Time Series [0.0]
  This work focuses on studying and analyzing the use of different techniques for data augmentation in time series for classification and regression problems. The proposed approach involves the use of diffusion probabilistic models, which have recently achieved successful results in the field of Image Processing. The results highlight the high utility of this methodology in creating synthetic data to train classification and regression models.
  arXiv  Detail & Related papers  (2023-12-09T11:37:07Z)
• CUTS: Neural Causal Discovery from Irregular Time-Series Data [27.06531262632836]
  Causal discovery from time-series data has been a central task in machine learning. We present CUTS, a neural Granger causal discovery algorithm to jointly impute unobserved data points and build causal graphs. Our approach constitutes a promising step towards applying causal discovery to real applications with non-ideal observations.
  arXiv  Detail & Related papers  (2023-02-15T04:16:34Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Automatic Data Augmentation via Invariance-Constrained Learning [94.27081585149836]
  Underlying data structures are often exploited to improve the solution of learning tasks. Data augmentation induces these symmetries during training by applying multiple transformations to the input data. This work tackles these issues by automatically adapting the data augmentation while solving the learning task.
  arXiv  Detail & Related papers  (2022-09-29T18:11:01Z)
• Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay [52.251188477192336]
  Few-shot class-incremental learning (FSCIL) has been proposed aiming to enable a deep learning system to incrementally learn new classes with limited data. We show through empirical results that adopting the data replay is surprisingly favorable. We propose using data-free replay that can synthesize data by a generator without accessing real data.
  arXiv  Detail & Related papers  (2022-07-22T17:30:51Z)
• Augmentation-Aware Self-Supervision for Data-Efficient GAN Training [68.81471633374393]
  Training generative adversarial networks (GANs) with limited data is challenging because the discriminator is prone to overfitting. We propose a novel augmentation-aware self-supervised discriminator that predicts the augmentation parameter of the augmented data. We compare our method with state-of-the-art (SOTA) methods using the class-conditional BigGAN and unconditional StyleGAN2 architectures.
  arXiv  Detail & Related papers  (2022-05-31T10:35:55Z)
• Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations [76.82124752950148]
  We develop a convenient gradient-based method for selecting the data augmentation. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
  arXiv  Detail & Related papers  (2022-02-22T02:51:11Z)
• Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations [86.5302150777089]
  Generative Adversarial Imputation Nets (GANs) and GAN-based techniques have attracted attention as unsupervised machine learning methods. We name our proposed method as Con Conval Generative Adversarial Imputation Nets (Conv-GAIN)
  arXiv  Detail & Related papers  (2021-11-03T03:50:48Z)
• Towards Synthetic Multivariate Time Series Generation for Flare Forecasting [5.098461305284216]
  One of the limiting factors in training data-driven, rare-event prediction algorithms is the scarcity of the events of interest. In this study, we explore the usefulness of the conditional generative adversarial network (CGAN) as a means to perform data-informed oversampling.
  arXiv  Detail & Related papers  (2021-05-16T22:23:23Z)
• Sequential IoT Data Augmentation using Generative Adversarial Networks [5.8010446129208155]
  Sequential data in industrial applications can be used to train and evaluate machine learning models. Since gathering representative amounts of data is difficult and time consuming, there is an incentive to generate it from a small ground truth. This paper investigates the possibility of using GANs in order to augment sequential Internet of Things (IoT) data.
  arXiv  Detail & Related papers  (2021-01-13T11:08:07Z)
• Approximate kNN Classification for Biomedical Data [1.1852406625172218]
  Single-cell RNA-seq (scRNA-seq) is an emerging DNA sequencing technology with promising capabilities but significant computational challenges. We propose the utilization of approximate nearest neighbor search algorithms for the task of kNN classification in scRNA-seq data.
  arXiv  Detail & Related papers  (2020-12-03T18:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.