A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learning

• URL: http://arxiv.org/abs/2508.10315v2
• Date: Mon, 13 Oct 2025 07:55:09 GMT
• Title: A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learning
• Authors: Keke Gai, Dongjue Wang, Jing Yu, Liehuang Zhu, Qi Wu,
• Abstract summary: We propose an FL backdoor defense framework, named CLIP-Fed, that utilizes the zero-shot learning capabilities of vision-language pre-training models.<n>Our scheme overcomes the limitations of Non-IID imposed on defense effectiveness by integrating pre-aggregation and post-aggregation defense strategies.
• Score: 43.847168319564844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Defending backdoor attacks in Federated Learning (FL) under heterogeneous client data distributions encounters limitations balancing effectiveness and privacy-preserving, while most existing methods highly rely on the assumption of homogeneous client data distributions or the availability of a clean serve dataset. In this paper, we propose an FL backdoor defense framework, named CLIP-Fed, that utilizes the zero-shot learning capabilities of vision-language pre-training models. Our scheme overcomes the limitations of Non-IID imposed on defense effectiveness by integrating pre-aggregation and post-aggregation defense strategies. CLIP-Fed aligns the knowledge of the global model and CLIP on the augmented dataset using prototype contrastive loss and Kullback-Leibler divergence, so that class prototype deviations caused by backdoor samples are ensured and the correlation between trigger patterns and target labels is eliminated. In order to balance privacy-preserving and coverage enhancement of the dataset against diverse triggers, we further construct and augment the server dataset via using the multimodal large language model and frequency analysis without any client samples. Extensive experiments on representative datasets evidence the effectiveness of CLIP-Fed. Comparing to other existing methods, CLIP-Fed achieves an average reduction in Attack Success Rate, {\em i.e.}, 2.03\% on CIFAR-10 and 1.35\% on CIFAR-10-LT, while improving average Main Task Accuracy by 7.92\% and 0.48\%, respectively. Our codes are available at https://anonymous.4open.science/r/CLIP-Fed.

Related papers

• Deep Leakage with Generative Flow Matching Denoiser [54.05993847488204]
  We introduce a new deep leakage (DL) attack that integrates a generative Flow Matching (FM) prior into the reconstruction process.<n>Our approach consistently outperforms state-of-the-art attacks across pixel-level, perceptual, and feature-based similarity metrics.
  arXiv  Detail & Related papers  (2026-01-21T14:51:01Z)
• GShield: Mitigating Poisoning Attacks in Federated Learning [2.6260952524631787]
  Federated Learning (FL) has recently emerged as a revolutionary approach to collaborative training Machine Learning models.<n>It enables decentralized model training while preserving data privacy, but its distributed nature makes it highly vulnerable to a severe attack known as Data Poisoning.<n>We present a novel defense mechanism called GShield, designed to detect and mitigate malicious and low-quality updates.
  arXiv  Detail & Related papers  (2025-12-22T11:29:28Z)
• Pigeon-SL: Robust Split Learning Framework for Edge Intelligence under Malicious Clients [53.496957000114875]
  We introduce Pigeon-SL, a novel scheme that guarantees at least one entirely honest cluster among M clients, even when up to N of them are adversarial.<n>In each global round, the access point partitions the clients into N+1 clusters, trains each cluster independently via vanilla SL, and evaluates their validation losses on a shared dataset.<n>Only the cluster with the lowest loss advances, thereby isolating and discarding malicious updates.
  arXiv  Detail & Related papers  (2025-08-04T09:34:50Z)
• FL-PLAS: Federated Learning with Partial Layer Aggregation for Backdoor Defense Against High-Ratio Malicious Clients [7.1383449614815415]
  Federated learning (FL) is gaining increasing attention as an emerging collaborative machine learning approach.<n>The fundamental algorithm of FL, Federated Averaging (FedAvg), is susceptible to backdoor attacks.<n>We propose a novel defense algorithm, FL-PLAS, which can effectively protect local models from backdoor attacks.
  arXiv  Detail & Related papers  (2025-05-17T14:16:47Z)
• Defending the Edge: Representative-Attention for Mitigating Backdoor Attacks in Federated Learning [7.808916974942399]
  heterogeneous edge devices produce diverse, non-independent, and identically distributed (non-IID) data.<n>We propose a novel representative-attention-based defense mechanism, named FeRA, to distinguish benign from malicious clients.<n>Our evaluation demonstrates FeRA's robustness across various FL scenarios, including challenging non-IID data distributions typical of edge devices.
  arXiv  Detail & Related papers  (2025-05-15T13:44:32Z)
• Client-side Gradient Inversion Against Federated Learning from Poisoning [59.74484221875662]
  Federated Learning (FL) enables distributed participants to train a global model without sharing data directly to a central server. Recent studies have revealed that FL is vulnerable to gradient inversion attack (GIA), which aims to reconstruct the original training samples. We propose Client-side poisoning Gradient Inversion (CGI), which is a novel attack method that can be launched from clients.
  arXiv  Detail & Related papers  (2023-09-14T03:48:27Z)
• G$^2$uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering [116.4277292854053]
  Federated Learning (FL) offers collaborative model training without data sharing. FL is vulnerable to backdoor attacks, where poisoned model weights lead to compromised system integrity. We present G$2$uardFL, a protective framework that reinterprets the identification of malicious clients as an attributed graph clustering problem.
  arXiv  Detail & Related papers  (2023-06-08T07:15:04Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks [52.26631767748843]
  We propose ROCLIP, the first effective method for robust pre-training multimodal vision-language models against targeted data poisoning and backdoor attacks. ROCLIP effectively breaks the association between poisoned image-caption pairs by considering a relatively large and varying pool of random captions. Our experiments show that ROCLIP renders state-of-the-art targeted data poisoning and backdoor attacks ineffective during pre-training CLIP models.
  arXiv  Detail & Related papers  (2023-03-13T04:49:46Z)
• FedCC: Robust Federated Learning against Model Poisoning Attacks [0.0]
  Federated learning is a distributed framework designed to address privacy concerns.<n>It introduces new attack surfaces, which are especially prone when data is non-Independently and Identically Distributed.<n>We present FedCC, a simple yet effective novel defense algorithm against model poisoning attacks.
  arXiv  Detail & Related papers  (2022-12-05T01:52:32Z)
• FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning [66.56240101249803]
  We study how hardening benign clients can affect the global model (and the malicious clients) We propose a trigger reverse engineering based defense and show that our method can achieve improvement with guarantee robustness. Our results on eight competing SOTA defense methods show the empirical superiority of our method on both single-shot and continuous FL backdoor attacks.
  arXiv  Detail & Related papers  (2022-10-23T22:24:03Z)
• Invariant Aggregator for Defending against Federated Backdoor Attacks [28.416262423174796]
  Federated learning enables training high-utility models across several clients without directly sharing their private data. As a downside, the federated setting makes the model vulnerable to various adversarial attacks in the presence of malicious clients. We propose an invariant aggregator that redirects the aggregated update to invariant directions that are generally useful.
  arXiv  Detail & Related papers  (2022-10-04T18:06:29Z)
• FL-Defender: Combating Targeted Attacks in Federated Learning [7.152674461313707]
  Federated learning (FL) enables learning a global machine learning model from local data distributed among a set of participating workers. FL is vulnerable to targeted poisoning attacks that negatively impact the integrity of the learned model. We propose textitFL-Defender as a method to combat FL targeted attacks.
  arXiv  Detail & Related papers  (2022-07-02T16:04:46Z)
• Backdoor Defense in Federated Learning Using Differential Testing and Outlier Detection [24.562359531692504]
  We propose DifFense, an automated defense framework to protect an FL system from backdoor attacks. Our detection method reduces the average backdoor accuracy of the global model to below 4% and achieves a false negative rate of zero.
  arXiv  Detail & Related papers  (2022-02-21T17:13:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.