Self-Supervision Improves Diffusion Models for Tabular Data Imputation

• URL: http://arxiv.org/abs/2407.18013v1
• Date: Thu, 25 Jul 2024 13:06:30 GMT
• Title: Self-Supervision Improves Diffusion Models for Tabular Data Imputation
• Authors: Yixin Liu, Thalaiyasingam Ajanthan, Hisham Husain, Vu Nguyen,
• Abstract summary: This paper introduces an advanced diffusion model named Self-supervised imputation Diffusion Model (SimpDM for brevity) To mitigate sensitivity to noise, we introduce a self-supervised alignment mechanism that aims to regularize the model, ensuring consistent and stable imputation predictions. We also introduce a carefully devised state-dependent data augmentation strategy within SimpDM, enhancing the robustness of the diffusion model when dealing with limited data.
• Score: 20.871219616589986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ubiquity of missing data has sparked considerable attention and focus on tabular data imputation methods. Diffusion models, recognized as the cutting-edge technique for data generation, demonstrate significant potential in tabular data imputation tasks. However, in pursuit of diversity, vanilla diffusion models often exhibit sensitivity to initialized noises, which hinders the models from generating stable and accurate imputation results. Additionally, the sparsity inherent in tabular data poses challenges for diffusion models in accurately modeling the data manifold, impacting the robustness of these models for data imputation. To tackle these challenges, this paper introduces an advanced diffusion model named Self-supervised imputation Diffusion Model (SimpDM for brevity), specifically tailored for tabular data imputation tasks. To mitigate sensitivity to noise, we introduce a self-supervised alignment mechanism that aims to regularize the model, ensuring consistent and stable imputation predictions. Furthermore, we introduce a carefully devised state-dependent data augmentation strategy within SimpDM, enhancing the robustness of the diffusion model when dealing with limited data. Extensive experiments demonstrate that SimpDM matches or outperforms state-of-the-art imputation methods across various scenarios.

Related papers

• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• Influence Functions for Scalable Data Attribution in Diffusion Models [52.92223039302037]
  Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges by developing an textitinfluence functions framework.
  arXiv  Detail & Related papers  (2024-10-17T17:59:02Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  We develop constrained diffusion models based on desired distributions informed by requirements. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• State-observation augmented diffusion model for nonlinear assimilation [6.682908186025083]
  We propose a novel data-driven assimilation algorithm based on generative models. Our State-Observation Augmented Diffusion (SOAD) model is designed to handle nonlinear physical and observational models more effectively.
  arXiv  Detail & Related papers  (2024-07-31T03:47:20Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks [0.0]
  We employ Bayesian neural networks to capture uncertainties in a reacting flow model. We demonstrate that BNN models can provide unique insights about the structure of uncertainty of the data-driven closure models. The efficacy of the model is demonstrated by a priori evaluation on a dataset consisting of a variety of flame conditions and fuels.
  arXiv  Detail & Related papers  (2024-02-28T22:19:55Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• MissDiff: Training Diffusion Models on Tabular Data with Missing Values [29.894691645801597]
  This work presents a unified and principled diffusion-based framework for learning from data with missing values. We first observe that the widely adopted "impute-then-generate" pipeline may lead to a biased learning objective. We prove the proposed method is consistent in learning the score of data distributions, and the proposed training objective serves as an upper bound for the negative likelihood in certain cases.
  arXiv  Detail & Related papers  (2023-07-02T03:49:47Z)
• Diffusion models for missing value imputation in tabular data [10.599563005836066]
  Missing value imputation in machine learning is the task of estimating the missing values in the dataset accurately using available information. We propose a diffusion model approach called "Conditional Score-based Diffusion Models for Tabular data" (CSDI_T) To effectively handle categorical variables and numerical variables simultaneously, we investigate three techniques: one-hot encoding, analog bits encoding, and feature tokenization.
  arXiv  Detail & Related papers  (2022-10-31T08:13:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.