BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation

• URL: http://arxiv.org/abs/2411.11006v1
• Date: Sun, 17 Nov 2024 09:01:55 GMT
• Title: BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation
• Authors: Haiyang Yu, Tian Xie, Jiaping Gui, Pengyang Wang, Ping Yi, Yue Wu,
• Abstract summary: We introduce BackdoorMBTI, the first backdoor learning toolkit and benchmark designed for multimodal evaluation. BackdoorMBTI provides a systematic backdoor learning pipeline, encompassing data processing, data poisoning, backdoor training, and evaluation.
• Score: 25.107072490555844
• License:
• Abstract: We introduce BackdoorMBTI, the first backdoor learning toolkit and benchmark designed for multimodal evaluation across three representative modalities from eleven commonly used datasets. BackdoorMBTI provides a systematic backdoor learning pipeline, encompassing data processing, data poisoning, backdoor training, and evaluation. The generated poison datasets and backdoor models enable detailed evaluation of backdoor defense methods. Given the diversity of modalities, BackdoorMBTI facilitates systematic evaluation across different data types. Furthermore, BackdoorMBTI offers a standardized approach to handling practical factors in backdoor learning, such as issues related to data quality and erroneous labels. We anticipate that BackdoorMBTI will expedite future research in backdoor defense methods within a multimodal context. Code is available at https://anonymous.4open.science/r/BackdoorMBTI-D6A1/README.md.

Related papers

• Expose Before You Defend: Unifying and Enhancing Backdoor Defenses via Exposed Models [68.40324627475499]
  We introduce a novel two-step defense framework named Expose Before You Defend. EBYD unifies existing backdoor defense methods into a comprehensive defense system with enhanced performance. We conduct extensive experiments on 10 image attacks and 6 text attacks across 2 vision datasets and 4 language datasets.
  arXiv  Detail & Related papers  (2024-10-25T09:36:04Z)
• BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning [41.66647711306716]
  We build a comprehensive benchmark of backdoor learning called BackdoorBench. We provide an integrated implementation of state-of-the-art (SOTA) backdoor learning algorithms. We conduct comprehensive evaluations with 5 poisoning ratios, based on 4 models and 4 datasets, leading to 11,492 pairs of attack-against-defense evaluations.
  arXiv  Detail & Related papers  (2024-07-29T09:57:03Z)
• Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency [15.643978173846095]
  Modern machine learning systems are vulnerable to backdoor poisoning attacks. We propose an automatic identification of backdoor data within a poisoned dataset. We show that our approach often surpasses the performance of current baselines in identifying backdoor data points.
  arXiv  Detail & Related papers  (2024-03-15T22:35:07Z)
• Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks [63.269788236474234]
  We propose to use model pairs on open-set classification tasks for detecting backdoors. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature.
  arXiv  Detail & Related papers  (2024-02-28T21:29:16Z)
• BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning [41.66647711306716]
  We build a comprehensive benchmark of backdoor learning called BackdoorBench. We provide an integrated implementation of state-of-the-art (SOTA) backdoor learning algorithms. We conduct comprehensive evaluations of 12 attacks against 16 defenses, with 5 poisoning ratios, based on 4 models and 4 datasets.
  arXiv  Detail & Related papers  (2024-01-26T17:03:38Z)
• Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypots [68.84056762301329]
  Recent research has exposed the susceptibility of pretrained language models (PLMs) to backdoor attacks. We propose and integrate a honeypot module into the original PLM to absorb backdoor information exclusively. Our design is motivated by the observation that lower-layer representations in PLMs carry sufficient backdoor features.
  arXiv  Detail & Related papers  (2023-10-28T08:21:16Z)
• BackdoorBench: A Comprehensive Benchmark of Backdoor Learning [57.932398227755044]
  Backdoor learning is an emerging and important topic of studying the vulnerability of deep neural networks (DNNs) Many pioneering backdoor attack and defense methods are being proposed successively or concurrently, in the status of a rapid arms race. We build a comprehensive benchmark of backdoor learning, called BackdoorBench.
  arXiv  Detail & Related papers  (2022-06-25T13:48:04Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Backdoor Learning: A Survey [75.59571756777342]
  Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs) Backdoor learning is an emerging and rapidly growing research area. This paper presents the first comprehensive survey of this realm.
  arXiv  Detail & Related papers  (2020-07-17T04:09:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.