The last Dance : Robust backdoor attack via diffusion models and bayesian approach

• URL: http://arxiv.org/abs/2402.05967v6
• Date: Mon, 23 Sep 2024 05:19:58 GMT
• Title: The last Dance : Robust backdoor attack via diffusion models and bayesian approach
• Authors: Orson Mengara,
• Abstract summary: Diffusion models are state-of-the-art deep learning generative models trained on the principle of learning forward and backward. We demonstrate the feasibility of backdoor attacks on audio transformers derived from Hugging Face, a popular framework in the world of artificial intelligence research.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion models are state-of-the-art deep learning generative models that are trained on the principle of learning forward and backward diffusion processes via the progressive addition of noise and denoising. In this paper, we aim to fool audio-based DNN models, such as those from the Hugging Face framework, primarily those that focus on audio, in particular transformer-based artificial intelligence models, which are powerful machine learning models that save time and achieve results faster and more efficiently. We demonstrate the feasibility of backdoor attacks (called `BacKBayDiffMod`) on audio transformers derived from Hugging Face, a popular framework in the world of artificial intelligence research. The backdoor attack developed in this paper is based on poisoning model training data uniquely by incorporating backdoor diffusion sampling and a Bayesian approach to the distribution of poisoned data.

Related papers

• UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Models [19.46962670935554]
  Diffusion Models are vulnerable to backdoor attacks. malicious attackers inject backdoors by poisoning some parts of the training samples. This poses a serious threat to the downstream users, who query the diffusion models through the API or directly download them from the internet.
  arXiv  Detail & Related papers  (2024-04-01T13:21:05Z)
• BEND: Bagging Deep Learning Training Based on Efficient Neural Network Diffusion [56.9358325168226]
  We propose a Bagging deep learning training algorithm based on Efficient Neural network Diffusion (BEND) Our approach is simple but effective, first using multiple trained model weights and biases as inputs to train autoencoder and latent diffusion model. Our proposed BEND algorithm can consistently outperform the mean and median accuracies of both the original trained model and the diffused model.
  arXiv  Detail & Related papers  (2024-03-23T08:40:38Z)
• Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [59.184980778643464]
  Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI) In this paper, we introduce an innovative technique called self-play fine-tuning for diffusion models (SPIN-Diffusion) Our approach offers an alternative to conventional supervised fine-tuning and RL strategies, significantly improving both model performance and alignment.
  arXiv  Detail & Related papers  (2024-02-15T18:59:18Z)
• Diffusion Models for Audio Restoration [22.385385150594185]
  We present here audio restoration algorithms based on diffusion models. We show that diffusion models can combine the best of both worlds and offer the opportunity to design audio restoration algorithms. We explain the diffusion formalism and its application to the conditional generation of clean audio signals.
  arXiv  Detail & Related papers  (2024-02-15T09:36:36Z)
• Investigating the Design Space of Diffusion Models for Speech Enhancement [17.914763947871368]
  Diffusion models are a new class of generative models that have shown outstanding performance in image generation literature. We show that the performance of previous diffusion-based speech enhancement systems cannot be attributed to the progressive transformation between the clean and noisy speech signals. We also show that a proper choice of preconditioning, training loss weighting, SDE and sampler allows to outperform a popular diffusion-based speech enhancement system.
  arXiv  Detail & Related papers  (2023-12-07T15:40:55Z)
• Leveraging Diffusion-Based Image Variations for Robust Training on Poisoned Data [26.551317580666353]
  Backdoor attacks pose a serious security threat for training neural networks. We propose a novel approach that enables model training on potentially poisoned datasets by utilizing the power of recent diffusion models.
  arXiv  Detail & Related papers  (2023-10-10T07:25:06Z)
• Unsupervised speech enhancement with diffusion-based generative models [0.0]
  We introduce an alternative approach that operates in an unsupervised manner, leveraging the generative power of diffusion models. We develop a posterior sampling methodology for speech enhancement by combining the learnt clean speech prior with a noise model for speech signal inference. We show promising results compared to a recent variational auto-encoder (VAE)-based unsupervised approach and a state-of-the-art diffusion-based supervised method.
  arXiv  Detail & Related papers  (2023-09-19T09:11:31Z)
• An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization [58.88327181933151]
  In this paper, we propose an efficient query-based membership inference attack (MIA) Experimental results indicate that the proposed method can achieve competitive performance with only two queries on both discrete-time and continuous-time diffusion models. To the best of our knowledge, this work is the first to study the robustness of diffusion models to MIA in the text-to-speech task.
  arXiv  Detail & Related papers  (2023-05-26T16:38:48Z)
• Diffusion Models as Masked Autoencoders [52.442717717898056]
  We revisit generatively pre-training visual representations in light of recent interest in denoising diffusion models. While directly pre-training with diffusion models does not produce strong representations, we condition diffusion models on masked input and formulate diffusion models as masked autoencoders (DiffMAE) We perform a comprehensive study on the pros and cons of design choices and build connections between diffusion models and masked autoencoders.
  arXiv  Detail & Related papers  (2023-04-06T17:59:56Z)
• How to Backdoor Diffusion Models? [74.43215520371506]
  This paper presents the first study on the robustness of diffusion models against backdoor attacks. We propose BadDiffusion, a novel attack framework that engineers compromised diffusion processes during model training for backdoor implantation. Our results call attention to potential risks and possible misuse of diffusion models.
  arXiv  Detail & Related papers  (2022-12-11T03:44:38Z)
• Diffusion Models in Vision: A Survey [80.82832715884597]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.