Continual Safety Alignment via Gradient-Based Sample Selection

• URL: http://arxiv.org/abs/2604.17215v1
• Date: Sun, 19 Apr 2026 02:52:33 GMT
• Title: Continual Safety Alignment via Gradient-Based Sample Selection
• Abstract summary: We investigate which training samples cause alignment drift through a data-centric lens.<n>We propose gradient-based sample selection that filters high-gradient samples during fine-tuning.<n>Our method is robust across selection ratios, task orderings, and diverse attack benchmarks.
• Score: 14.306119791052575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models require continuous adaptation to new tasks while preserving safety alignment. However, fine-tuning on even benign data often compromises safety behaviors, including refusal of harmful requests, truthfulness, and commonsense reasoning. We investigate which training samples cause alignment drift through a data-centric lens. Our empirical analysis shows samples contribute unequally: high-gradient samples cause greater safety degradation and drive models toward pretrained distributions, while moderate-gradient samples enable task learning with minimal alignment loss. We propose gradient-based sample selection that filters high-gradient samples during fine-tuning. Across multiple model families on continual domain tasks, our method substantially improves alignment preservation while maintaining competitive task performance, without requiring curated safe data or architectural modifications. Our method is robust across selection ratios, task orderings, and diverse attack benchmarks.

Related papers

• Shieldstral [183.97744446582863]
  Shieldstral is a policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7$times$ its size on text safety benchmarks.<n>We present a data construction recipe, covering curation and generation of approximately 54.1M samples and a fine-grained evaluation set to evaluate policy adaptability.
  arXiv  Detail & Related papers  (2026-07-28T15:27:53Z)
• Gradient-Guided Reward Optimization for Inference-time Alignment [22.28081683932275]
  We introduce a lightweight inference-time method that performs targeted, minimal intervention during decoding via gradient guidance.<n>Experiments show that GGRO consistently improves inference-time alignment across safety, helpfulness, and reasoning benchmarks.<n>It also increases coverage of high-quality responses and robustness to reward hacking, with minimal computational overhead.
  arXiv  Detail & Related papers  (2026-06-08T15:33:13Z)
• Token-level Data Selection for Safe LLM Fine-tuning [15.039068315115372]
  Fine-tuning large language models (LLMs) on custom datasets has become a standard approach for adapting these models to specific domains and applications.<n>Recent studies have shown that such fine-tuning can lead to significant degradation in the model's safety.<n>We propose a novel framework that quantifies the safety risk of each token by measuring the loss difference between a safety-degraded model and a utility-oriented model.
  arXiv  Detail & Related papers  (2026-03-01T16:52:05Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective [56.31306558218838]
  We show that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts.<n>We propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance.<n>Our experiments on the Llama families across multiple datasets demonstrate that safety problems can largely be avoided without specialized interventions.
  arXiv  Detail & Related papers  (2025-08-17T23:46:36Z)
• Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning [5.778730972088573]
  Gradient-aligned samples are selected for their potential to reduce interference by re-aligning gradients.<n> gradient-aligned samples are incorporated to reinforce stable, shared representations across tasks.<n>Our approach increases the diversity among retrieved instances and achieves superior alignment in parameter space.
  arXiv  Detail & Related papers  (2024-12-19T01:08:09Z)
• Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection [82.43311784594384]
  Real-world datasets contain not only noisy labels but also class imbalance. We propose a simple yet effective method to address noisy labels in imbalanced datasets.
  arXiv  Detail & Related papers  (2024-02-17T10:34:53Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Continual Test-time Domain Adaptation via Dynamic Sample Selection [38.82346845855512]
  This paper proposes a Dynamic Sample Selection (DSS) method for Continual Test-time Domain Adaptation (CTDA) We apply joint positive and negative learning on both high- and low-quality samples to reduce the risk of using wrong information. Our approach is also evaluated in the 3D point cloud domain, showcasing its versatility and potential for broader applicability.
  arXiv  Detail & Related papers  (2023-10-05T06:35:21Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.