GradSentry: Gradient Spectral Entropy for Backdoor Sample Filtering in Large Language Model Fine-Tuning

• URL: http://arxiv.org/abs/2605.26574v1
• Date: Tue, 26 May 2026 05:48:01 GMT
• Title: GradSentry: Gradient Spectral Entropy for Backdoor Sample Filtering in Large Language Model Fine-Tuning
• Abstract summary: Fine-tuning Large Language Models with untrusted data exposes models to backdoor attacks.<n>We propose GradSentry, a backdoor sample filtering method based on the spectral entropy of per-sample gradients.
• Score: 29.78266005682094
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning Large Language Models with untrusted data exposes models to backdoor attacks, where poisoned samples cause targeted misbehavior. Existing sample-filtering defenses rely on clustering, which requires sufficient data and can fail at extreme poison ratios. We propose GradSentry ({Grad}ient {Sentry}), a backdoor sample filtering method based on the spectral entropy of per-sample gradients. Our key finding is that poisoned samples produce gradients with higher spectral entropy compared to clean samples. GradSentry captures output-altering backdoor signatures using per-sample gradient spectra, avoiding pairwise sample comparisons and clustering during feature construction. Importantly, our method is training-agnostic: it works for both parameter-efficient fine-tuning methods like LoRA and full-parameter tuning, as the gradient analysis operates independently of which parameters are being updated during training. GradSentry requires no clustering, operates effectively across all poison ratios (1%--90%), and introduces minimal computational overhead (20-50ms per sample for 7B model). Evaluation on four QA datasets and four attack types demonstrates the effectiveness of spectral entropy for backdoor detection. Code is available at https://github.com/dongdongzhaoUP/GradSentry.

Related papers

• EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy [66.59724477993339]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks.<n>We propose EntropyScan, a lightweight and trigger-agnostic method for model-level backdoor detection in LVLMs.<n>Our code will be publicly available soon.
  arXiv  Detail & Related papers  (2026-05-15T08:01:32Z)
• Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models [21.927099943522734]
  We propose a backdoor sample detection method based on perturbatiotextbfN discrtextbfEpancy consistextbfTency textbfEvaluation (NETE)<n>In the detection process, it only requires an off-the-shelf pre-trained model to compute the log probability of samples.<n>Our method is based on the interesting phenomenon that the change in perturbation discrepancy for backdoor samples is smaller than that for clean samples.
  arXiv  Detail & Related papers  (2025-08-30T06:35:32Z)
• Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios [65.97836905826145]
  scarcity of data in various scenarios, such as medical, industry and autonomous driving, leads to model overfitting and dataset imbalance.<n>We propose Crucial-Diff, a domain-agnostic framework designed to synthesize crucial samples.<n>Our framework generates diverse, high-quality training data, achieving a pixel-level AP of 83.63% and an F1-MAX of 78.12% on MVTec.
  arXiv  Detail & Related papers  (2025-07-14T04:41:38Z)
• Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models [12.519879298717104]
  We propose a stealthy backdoor sample detection method based on Reference-Filtration and Tfidf-Clustering mechanisms.<n>Experiments on two machine translation datasets and one QA dataset demonstrate that RFTC outperforms baselines in backdoor detection and model performance.
  arXiv  Detail & Related papers  (2025-05-29T02:49:29Z)
• Distributional Diffusion Models with Scoring Rules [83.38210785728994]
  Diffusion models generate high-quality synthetic data.<n> generating high-quality outputs requires many discretization steps.<n>We propose to accomplish sample generation by learning the posterior em distribution of clean data samples.
  arXiv  Detail & Related papers  (2025-02-04T16:59:03Z)
• CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling [63.07948989346385]
  Federated learning collaboratively trains a neural network on a global server.<n>Each local client receives the current global model weights and sends back parameter updates (gradients) based on its local private data.<n>Existing gradient inversion attacks can exploit this vulnerability to recover private training instances from a client's gradient vectors.<n>We present a novel defense tailored for large neural network models.
  arXiv  Detail & Related papers  (2025-01-27T01:06:23Z)
• Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining [16.76094864625033]
  Backdoor attacks are significant security threats to generative large language models (LLMs)<n>GraCeFul uses sample-wise gradients in the frequency space to identify backdoor samples without requiring retraining LLMs.<n>GraCeFul exhibits remarkable computational efficiency, achieving nearly 100% recall and F1 scores in identifying backdoor samples.
  arXiv  Detail & Related papers  (2024-12-03T13:43:36Z)
• Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing [10.48229619478838]
  Model stealing (MS) involves querying and observing the output of a machine learning model to steal its capabilities. Superpixel Sample Gradient stealing (SPSG) for model stealing under the constraint of limited real samples.
  arXiv  Detail & Related papers  (2024-05-18T08:38:43Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Activation Gradient based Poisoned Sample Detection Against Backdoor Attacks [35.42528584450334]
  We develop an innovative poisoned sample detection approach, called Activation Gradient based Poisoned sample Detection (AGPD) First, we calculate GCDs of all classes from the model trained on the untrustworthy dataset. Then, we identify the target class(es) based on the difference on GCD dispersion between target and clean classes. Last, we filter out poisoned samples within the identified target class(es) based on the clear separation between poisoned and clean samples.
  arXiv  Detail & Related papers  (2023-12-11T09:17:33Z)
• XGBD: Explanation-Guided Graph Backdoor Detection [21.918945251903523]
  Backdoor attacks pose a significant security risk to graph learning models. We propose an explanation-guided backdoor detection method to take advantage of the topological information.
  arXiv  Detail & Related papers  (2023-08-08T17:10:23Z)
• CrowdGuard: Federated Backdoor Detection in Federated Learning [39.58317527488534]
  This paper presents a novel defense mechanism, CrowdGuard, that effectively mitigates backdoor attacks in Federated Learning. CrowdGuard employs a server-located stacked clustering scheme to enhance its resilience to rogue client feedback. The evaluation results demonstrate that CrowdGuard achieves a 100% True-Positive-Rate and True-Negative-Rate across various scenarios.
  arXiv  Detail & Related papers  (2022-10-14T11:27:49Z)
• Learning Summary Statistics for Bayesian Inference with Autoencoders [58.720142291102135]
  We use the inner dimension of deep neural network based Autoencoders as summary statistics. To create an incentive for the encoder to encode all the parameter-related information but not the noise, we give the decoder access to explicit or implicit information that has been used to generate the training data.
  arXiv  Detail & Related papers  (2022-01-28T12:00:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.