TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering

• URL: http://arxiv.org/abs/2601.03300v1
• Date: Tue, 06 Jan 2026 03:02:20 GMT
• Title: TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering
• Abstract summary: We present TRYLOCK, the first defense-in-depth architecture that combines four mechanisms across the inference stack.<n>On Mistral-7B-Instruct evaluated against a 249-prompt attack set, TRYLOCK achieves 88.0% relative ASR reduction.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models remain vulnerable to jailbreak attacks, and single-layer defenses often trade security for usability. We present TRYLOCK, the first defense-in-depth architecture that combines four heterogeneous mechanisms across the inference stack: weight-level safety alignment via DPO, activation-level control via Representation Engineering (RepE) steering, adaptive steering strength selected by a lightweight sidecar classifier, and input canonicalization to neutralize encoding-based bypasses. On Mistral-7B-Instruct evaluated against a 249-prompt attack set spanning five attack families, TRYLOCK achieves 88.0% relative ASR reduction (46.5% to 5.6%), with each layer contributing unique coverage: RepE blocks 36% of attacks that bypass DPO alone, while canonicalization catches 14% of encoding attacks that evade both. We discover a non-monotonic steering phenomenon -- intermediate strength (alpha=1.0) degrades safety below baseline -- and provide mechanistic hypotheses explaining RepE-DPO interference. The adaptive sidecar reduces over-refusal from 60% to 48% while maintaining identical attack defense, demonstrating that security and usability need not be mutually exclusive. We release all components -- trained adapters, steering vectors, sidecar classifier, preference pairs, and complete evaluation methodology -- enabling full reproducibility.

Related papers

• Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration [37.700303596218184]
  Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA)<n>We introduce Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning.<n>We prove a spectral bound formalizing this effect and evaluate DDO across six model families, achieving 10% ASR under standard RFA.
  arXiv  Detail & Related papers  (2026-09-14T18:36:35Z)
• MechAudit-40: White-Box Auditing across 40 LLM Attack Mechanisms [16.312485460260056]
  MechAudit-40 is a systematic evaluation of 40 attack mechanisms across five open-weight model architectures.<n>Attacks induce structured multi-depth trajectories rather than isolated layer spikes.<n> MechAudit detects 81.1% of held-out attack executions at a 0.70% false-positive rate and maintains 78.1% recall when an entire functional category is withheld.
  arXiv  Detail & Related papers  (2026-09-06T13:57:41Z)
• AlcaTRAz - Anchored Tree-Rule Defense Against Jailbreaks [0.08699280339422538]
  Large language models (LLMs) are vulnerable to jailbreak attacks that bypass safety alignment through carefully crafted prompts.<n>We propose AlcaTRAz, a prompt-level defense based on rule trees that operates exclusively on the input text.<n>We evaluate the proposed method across 33 open-weight models, 22 jailbreak attack types, and a benchmark of short, single-turn benign questions.
  arXiv  Detail & Related papers  (2026-09-03T11:30:08Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• Belief-Space Quantum-Inspired Reinforcement Learning for Partially Observable Autonomous Cyber Defense in the Internet of Vehicles [1.1411682827396572]
  Internet of Vehicles (IoV) faces a dynamic, adversarial security environment where attackers adapt to defenses.<n>We formulate IoV security as a sequential attacker-defender interaction and model defense as a reinforcement learning problem under partial observability.<n>We propose Quantum Belief-Integrated Reinforcement Defense (Q-BIRD) using quantum-inspired belief representation to encode defender uncertainty about hidden attacker intent.
  arXiv  Detail & Related papers  (2026-06-05T19:20:38Z)
• Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy [0.7212939068975618]
  We find that pretrained base models exhibit the same substitution pattern as their Instruct variants, averaging higher yield than Instruct.<n> patching above this window restores 96% of the clean-to-pressured P(correct) gap.<n>Two converging activation-space interventions show that pressure suppresses clean-reasoning features rather than activating a new sycophancy circuit.
  arXiv  Detail & Related papers  (2026-05-13T04:45:08Z)
• Efficient Refusal Ablation in LLM through Optimal Transport [30.0180859405821]
  Safety-aligned language models refuse harmful requests through learned refusal behaviors encoded in their internal representations.<n>Recent activation-based jailbreaking methods circumvent these safety mechanisms by applying projections to remove refusal directions.<n>We introduce a principled framework based on optimal transport theory that transforms the entire distribution of harmful activations to match harmless ones.
  arXiv  Detail & Related papers  (2026-03-04T18:19:50Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• What Matters For Safety Alignment? [38.86339753409445]
  This paper presents a comprehensive empirical study on the safety alignment capabilities of AI systems.<n>We systematically investigate and compare the influence of six critical intrinsic model characteristics and three external attack techniques.<n>We identify the LRMs GPT-OSS-20B, Qwen3-Next-80B-A3B-Thinking, and GPT-OSS-120B as the top-three safest models.
  arXiv  Detail & Related papers  (2026-01-07T12:31:52Z)
• Security Without Detection: Economic Denial as a Primitive for Edge and IoT Defense [0.0]
  Economic Denial Security (EDS) is a detection-independent framework that makes attacks economically exploit by exploiting a fundamental asymmetry.<n>EDS composes four mechanisms adaptive computational decoy-driven interaction puzzles, entropy-driven interaction puzzles, temporal stretching, and bandwidth taxation.<n>EDS provides detection-independent protection suitable for resource-constrained environments where traditional approaches fail.
  arXiv  Detail & Related papers  (2025-12-29T20:28:46Z)
• Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation [13.509767769174422]
  Safety alignment is critical for training large language models to refuse harmful requests.<n>Low-Rank Adaptation (LoRA) consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks.<n>We propose SAILS (Safety Alignment via Interpretable Low-rank Subspace) to address this gap.
  arXiv  Detail & Related papers  (2025-12-29T07:39:49Z)
• Multi-Faceted Attack: Exposing Cross-Model Vulnerabilities in Defense-Equipped Vision-Language Models [54.61181161508336]
  We introduce Multi-Faceted Attack (MFA), a framework that exposes general safety vulnerabilities in leading defense-equipped Vision-Language Models (VLMs)<n>The core component of MFA is the Attention-Transfer Attack (ATA), which hides harmful instructions inside a meta task with competing objectives.<n>MFA achieves a 58.5% success rate and consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2025-11-20T07:12:54Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models [64.47869632167284]
  Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.<n>This sequential approach creates a mismatch -- attackers overfit to obsolete defenses, while defenders perpetually lag behind emerging threats.<n>We propose Self-RedTeam, an online self-play reinforcement learning algorithm where an attacker and defender agent co-evolve through continuous interaction.
  arXiv  Detail & Related papers  (2025-06-09T06:35:12Z)
• Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers [14.262681970049172]
  Large Reasoning Models (LRMs) have demonstrated superior logical capabilities compared to traditional Large Language Models (LLMs)<n>SEAL is a novel jailbreak attack that targets LRMs through an adaptive encryption pipeline designed to override their reasoning processes and evade potential adaptive alignment.<n>SEAL achieves an attack success rate of 80.8% on GPT o4-mini, outperforming state-of-the-art baselines by a significant margin of 27.2%.
  arXiv  Detail & Related papers  (2025-05-22T05:19:42Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.