Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs

• URL: http://arxiv.org/abs/2604.15780v1
• Date: Fri, 17 Apr 2026 07:37:41 GMT
• Title: Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs
• Abstract summary: Current alignment methods do not explicitly remove the unsafeworks that trigger harmful outputs.<n>In this work, we introduce a resource-efficient pruning framework that directly identifies and removes parameters associated with unsafe behaviors.<n> Empirical evaluations on ML models show substantial reductions in unsafe generations and improved robustness against jailbreak attacks, with minimal utility loss.
• Score: 23.758453206408102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models are increasingly deployed in real-world applications, but even aligned models such as Mistral and LLaVA still exhibit unsafe behaviors inherited from pre-training. Current alignment methods like SFT and RLHF primarily encourage models to generate preferred responses, but do not explicitly remove the unsafe subnetworks that trigger harmful outputs. In this work, we introduce a resource-efficient pruning framework that directly identifies and removes parameters associated with unsafe behaviors while preserving model utility. Our method employs a gradient-free attribution mechanism, requiring only modest GPU resources, and generalizes across architectures and quantized variants. Empirical evaluations on ML models show substantial reductions in unsafe generations and improved robustness against jailbreak attacks, with minimal utility loss. From the perspective of the Lottery Ticket Hypothesis, our results suggest that ML models contain "unsafe tickets" responsible for harmful behaviors, and pruning reveals "safety tickets" that maintain performance while aligning outputs. This provides a lightweight, post-hoc alignment strategy suitable for deployment in resource-constrained settings.

Related papers

• EVA: Editing for Versatile Alignment against Jailbreaks [85.18411558566851]
  EVA (Editing for Versatile Alignment against Jailbreaks) is a novel framework that pioneers the application of direct model editing for safety alignment.<n> EVA identifies and surgically edits specific neurons responsible for the model's susceptibility to harmful instructions.<n>It effectively neutralizes harmful behaviors without compromising the model's general reasoning capabilities.
  arXiv  Detail & Related papers  (2026-05-14T12:16:10Z)
• Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks [4.726777092009554]
  Open-weight language models can be rendered unsafe through several distinct interventions.<n>We study behavioral and mechanistic properties of jailbroken models across three unsafe routes.<n>All three routes achieve near-ceiling harmful compliance, but they diverge once we move beyond direct harmfulness.
  arXiv  Detail & Related papers  (2026-04-20T17:01:27Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models [71.11171576590134]
  Vision-language models can be easily induced by visual jailbreak attacks to generate harmful content.<n>We propose NullSteer, a null-space projected activation defense framework.<n>We show that NullSteer significantly reduces harmful outputs under various jailbreak attacks.
  arXiv  Detail & Related papers  (2026-03-23T15:23:23Z)
• THINKSAFE: Self-Generated Safety Alignment for Reasoning Models [60.10077024249373]
  We propose ThinkSafe, a framework that restores safety alignment without external teachers.<n>Our key insight is that while compliance suppresses safety mechanisms, models often retain latent knowledge to identify harm.<n> Experiments on DeepSeek-R1-Distill and Qwen3 show ThinkSafe significantly improves safety while preserving reasoning proficiency.
  arXiv  Detail & Related papers  (2026-01-30T16:31:02Z)
• Patching LLM Like Software: A Lightweight Method for Improving Safety Policy in Large Language Models [63.54707418559388]
  We propose patching for large language models (LLMs) like software versions.<n>Our method enables rapid remediation by prepending a compact, learnable prefix to an existing model.
  arXiv  Detail & Related papers  (2025-11-11T17:25:44Z)
• SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection [16.38885847999291]
  Large Language Models (LLMs) with safe-alignment training are powerful instruments with robust language comprehension capabilities.<n>We introduce a novel white-box jailbreak method, SABER, which connects two intermediate layers $s$ and $e$ such that $s e$, through a residual connection.<n>Our approach achieves a 51% improvement over the best-performing baseline on the HarmBench test set.
  arXiv  Detail & Related papers  (2025-09-19T15:10:19Z)
• Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs [15.23174472320989]
  Large Language Models (LLMs) are central to many contemporary AI applications.<n>Recent works in eXplainable AI (XAI) suggest that interpretability can also enable model compression.
  arXiv  Detail & Related papers  (2025-06-16T17:38:36Z)
• STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models [31.35788474507371]
  Large Language Models (LLMs) have become increasingly vulnerable to jailbreak attacks.<n>We present STShield, a lightweight framework for real-time jailbroken judgement.
  arXiv  Detail & Related papers  (2025-03-23T04:23:07Z)
• Do LLMs Understand the Safety of Their Inputs? Training-Free Moderation via Latent Prototypes [1.0779346838250028]
  Latent Prototype Moderator (LPM) is a training-free moderation method that uses Mahalanobis distance in latent space to assess input safety.<n>LPM matches or exceeds state-of-the-art guard models across multiple safety benchmarks.
  arXiv  Detail & Related papers  (2025-02-22T10:31:50Z)
• DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing [62.43110639295449]
  Large Language Models (LLMs) are widely applied in decision making, but their deployment is threatened by jailbreak attacks.<n>Delman is a novel approach leveraging direct model editing for precise, dynamic protection against jailbreak attacks.<n>Delman directly updates a minimal set of relevant parameters to neutralize harmful behaviors while preserving the model's utility.
  arXiv  Detail & Related papers  (2025-02-17T10:39:21Z)
• STAIR: Improving Safety Alignment with Introspective Reasoning [44.780098674618614]
  We propose STAIR, a framework that integrates SafeTy Alignment with Itrospective Reasoning.<n>We show that STAIR effectively mitigates harmful outputs while better preserving helpfulness, compared to instinctive alignment strategies.<n>With test-time scaling, STAIR achieves a safety performance comparable to Claude-3.5 against popular jailbreak attacks.
  arXiv  Detail & Related papers  (2025-02-04T15:02:55Z)
• Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities [49.09703018511403]
  Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks.<n>Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system.<n>We propose evaluating LLMs with model tampering attacks which allow for modifications to latent activations or weights.
  arXiv  Detail & Related papers  (2025-02-03T18:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.