Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss

• URL: http://arxiv.org/abs/2403.16728v1
• Date: Mon, 25 Mar 2024 13:02:43 GMT
• Title: Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss
• Authors: Artem Khrapov, Vadim Popov, Tasnima Sadekova, Assel Yermekova, Mikhail Kudinov,
• Abstract summary: We propose an alternative diffusion loss function, which can preserve the high quality of generated data while being robust to outliers. We show that pseudo-Huber loss with the time-dependent parameter exhibits better performance on corrupted datasets in both image and audio domains.
• Score: 5.539965805440292
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are known to be vulnerable to outliers in training data. In this paper we study an alternative diffusion loss function, which can preserve the high quality of generated data like the original squared $L_{2}$ loss while at the same time being robust to outliers. We propose to use pseudo-Huber loss function with a time-dependent parameter to allow for the trade-off between robustness on the most vulnerable early reverse-diffusion steps and fine details restoration on the final steps. We show that pseudo-Huber loss with the time-dependent parameter exhibits better performance on corrupted datasets in both image and audio domains. In addition, the loss function we propose can potentially help diffusion models to resist dataset corruption while not requiring data filtering or purification compared to conventional training algorithms.

Related papers

• Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions [8.666879925570331]
  Real-world offline datasets are often subject to data corruptions due to sensor failures or malicious attacks. Existing methods struggle to learn robust agents under high uncertainty caused by corrupted data. We propose a novel robust variational Bayesian inference for offline RL (TRACER)
  arXiv  Detail & Related papers  (2024-11-01T09:28:24Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data [74.2507346810066]
  Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data.
  arXiv  Detail & Related papers  (2024-03-20T14:22:12Z)
• Recovering high-quality FODs from a reduced number of diffusion-weighted images using a model-driven deep learning architecture [0.0]
  We propose a model-driven deep learning FOD reconstruction architecture. It ensures intermediate and output FODs produced by the network are consistent with the input DWI signals. Our results show that the model-based deep learning architecture achieves competitive performance compared to a state-of-the-art FOD super-resolution network, FOD-Net.
  arXiv  Detail & Related papers  (2023-07-28T02:47:34Z)
• DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness [78.98998551326812]
  We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis. We show that the method can exploit the multi-modality features from related sensors and also learn from history time-series dynamics to reconstruct the data under extreme missingness.
  arXiv  Detail & Related papers  (2022-09-26T21:59:14Z)
• Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders [137.1060633388405]
  Diffusion-based generative models learn how to generate the data by inferring a reverse diffusion chain. We propose a faster and cheaper approach that adds noise not until the data become pure random noise. We show that the proposed model can be cast as an adversarial auto-encoder empowered by both the diffusion process and a learnable implicit prior.
  arXiv  Detail & Related papers  (2022-02-19T20:18:49Z)
• Mean-Shifted Contrastive Loss for Anomaly Detection [34.97652735163338]
  We propose a new loss function which can overcome failure modes of both center-loss and contrastive-loss methods. Our improvements yield a new anomaly detection approach, based on $textitMean-Shifted Contrastive Loss$. Our method achieves state-of-the-art anomaly detection performance on multiple benchmarks including $97.5%$ ROC-AUC.
  arXiv  Detail & Related papers  (2021-06-07T17:58:03Z)
• Continual Learning for Fake Audio Detection [62.54860236190694]
  This paper proposes detecting fake without forgetting, a continual-learning-based method, to make the model learn new spoofing attacks incrementally. Experiments are conducted on the ASVspoof 2019 dataset.
  arXiv  Detail & Related papers  (2021-04-15T07:57:05Z)
• Time-Series Imputation with Wasserstein Interpolation for Optimal Look-Ahead-Bias and Variance Tradeoff [66.59869239999459]
  In finance, imputation of missing returns may be applied prior to training a portfolio optimization model. There is an inherent trade-off between the look-ahead-bias of using the full data set for imputation and the larger variance in the imputation from using only the training data. We propose a Bayesian posterior consensus distribution which optimally controls the variance and look-ahead-bias trade-off in the imputation.
  arXiv  Detail & Related papers  (2021-02-25T09:05:35Z)
• Federated Survival Analysis with Discrete-Time Cox Models [0.46331617589391827]
  We build machine learning models from decentralized datasets located in different centers with federated learning (FL) We show that the resulting model may suffer from important performance loss in some adverse settings. Using this approach, we train survival models using standard FL techniques on synthetic data, as well as real-world datasets from The Cancer Genome Atlas (TCGA)
  arXiv  Detail & Related papers  (2020-06-16T08:53:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.