Artificial Inductive Bias for Synthetic Tabular Data Generation in Data-Scarce Scenarios

• URL: http://arxiv.org/abs/2407.03080v1
• Date: Wed, 3 Jul 2024 12:53:42 GMT
• Title: Artificial Inductive Bias for Synthetic Tabular Data Generation in Data-Scarce Scenarios
• Authors: Patricia A. Apellániz, Ana Jiménez, Borja Arroyo Galende, Juan Parras, Santiago Zazo,
• Abstract summary: We propose a novel methodology for generating realistic and reliable synthetic data with Deep Generative Models (DGMs) in limited real-data environments. Our approach proposes several ways to generate an artificial inductive bias in a DGM through transfer learning and meta-learning techniques. We validate our approach using two state-of-the-art DGMs, namely, a Variational Autoencoder and a Generative Adversarial Network, to show that our artificial inductive bias fuels superior synthetic data quality.
• Score: 8.062368743143388
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: While synthetic tabular data generation using Deep Generative Models (DGMs) offers a compelling solution to data scarcity and privacy concerns, their effectiveness relies on substantial training data, often unavailable in real-world applications. This paper addresses this challenge by proposing a novel methodology for generating realistic and reliable synthetic tabular data with DGMs in limited real-data environments. Our approach proposes several ways to generate an artificial inductive bias in a DGM through transfer learning and meta-learning techniques. We explore and compare four different methods within this framework, demonstrating that transfer learning strategies like pre-training and model averaging outperform meta-learning approaches, like Model-Agnostic Meta-Learning, and Domain Randomized Search. We validate our approach using two state-of-the-art DGMs, namely, a Variational Autoencoder and a Generative Adversarial Network, to show that our artificial inductive bias fuels superior synthetic data quality, as measured by Jensen-Shannon divergence, achieving relative gains of up to 50\% when using our proposed approach. This methodology has broad applicability in various DGMs and machine learning tasks, particularly in areas like healthcare and finance, where data scarcity is often a critical issue.

Related papers

• Exploring the Landscape for Generative Sequence Models for Specialized Data Synthesis [0.0]
  This paper introduces a novel approach that leverages three generative models of varying complexity to synthesize Malicious Network Traffic. Our approach transforms numerical data into text, re-framing data generation as a language modeling task. Our method surpasses state-of-the-art generative models in producing high-fidelity synthetic data.
  arXiv  Detail & Related papers  (2024-11-04T09:51:10Z)
• Differentially Private Tabular Data Synthesis using Large Language Models [6.6376578496141585]
  This paper introduces DP-LLMTGen -- a novel framework for differentially private tabular data synthesis. DP-LLMTGen models sensitive datasets using a two-stage fine-tuning procedure. It generates synthetic data through sampling the fine-tuned LLMs.
  arXiv  Detail & Related papers  (2024-06-03T15:43:57Z)
• A Comprehensive Survey on Data Augmentation [55.355273602421384]
  Data augmentation is a technique that generates high-quality artificial data by manipulating existing data samples. Existing literature surveys only focus on a certain type of specific modality data. We propose a more enlightening taxonomy that encompasses data augmentation techniques for different common data modalities.
  arXiv  Detail & Related papers  (2024-05-15T11:58:08Z)
• Generative AI for Synthetic Data Generation: Methods, Challenges and the Future [12.506811635026907]
  The recent surge in research focused on generating synthetic data from large language models (LLMs) This paper delves into advanced technologies that leverage these gigantic LLMs for the generation of task-specific training data.
  arXiv  Detail & Related papers  (2024-03-07T03:38:44Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark [56.8042116967334]
  Synthetic data serves as an alternative in training machine learning models. ensuring that synthetic data mirrors the complex nuances of real-world data is a challenging task. This paper explores the potential of integrating data-centric AI techniques to guide the synthetic data generation process.
  arXiv  Detail & Related papers  (2023-10-25T20:32:02Z)
• DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation [1.4418363806859886]
  We present Differentially-Private TaBular AutoRegressive Transformer (DP-TBART), a transformer-based autoregressive model that maintains differential privacy. We provide a theoretical framework for understanding the limitations of marginal-based approaches and where deep learning-based approaches stand to contribute most.
  arXiv  Detail & Related papers  (2023-07-19T19:40:21Z)
• TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series [61.436361263605114]
  Time series data are often scarce or highly sensitive, which precludes the sharing of data between researchers and industrial organizations. We introduce Time Series Generative Modeling (TSGM), an open-source framework for the generative modeling of synthetic time series.
  arXiv  Detail & Related papers  (2023-05-19T10:11:21Z)
• FairGen: Fair Synthetic Data Generation [0.3149883354098941]
  We propose a pipeline to generate fairer synthetic data independent of the GAN architecture. We claim that while generating synthetic data most GANs amplify bias present in the training data but by removing these bias inducing samples, GANs essentially focuses more on real informative samples.
  arXiv  Detail & Related papers  (2022-10-24T08:13:47Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators [74.16405337436213]
  We propose Gradient-sanitized Wasserstein Generative Adrial Networks (GS-WGAN) GS-WGAN allows releasing a sanitized form of sensitive data with rigorous privacy guarantees. We find our approach consistently outperforms state-of-the-art approaches across multiple metrics.
  arXiv  Detail & Related papers  (2020-06-15T10:01:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.