CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoders

• URL: http://arxiv.org/abs/2604.01604v1
• Date: Thu, 02 Apr 2026 04:28:11 GMT
• Title: CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoders
• Abstract summary: We propose a circuit-guided refusal feature selection framework that ranks features by their influence on the model's refusal-compliance decision.<n>On Gemma-3-1B-it, CRaFT improves attack success rate (ASR) from 6.7% to 48.2% and outperforms baseline methods.
• Score: 10.526176863220988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As safety concerns around large language models (LLMs) grow, understanding the internal mechanisms underlying refusal behavior has become increasingly important. Recent work has studied this behavior by identifying internal features associated with refusal and manipulating them to induce compliance with harmful requests. However, existing refusal feature selection methods rely on how strongly features activate on harmful prompts, which tends to capture superficial signals rather than the causal factors underlying the refusal decision. We propose CRaFT, a circuit-guided refusal feature selection framework that ranks features by their influence on the model's refusal-compliance decision using prompts near the refusal boundary. On Gemma-3-1B-it, CRaFT improves attack success rate (ASR) from 6.7% to 48.2% and outperforms baseline methods across multiple jailbreak benchmarks. These results suggest that circuit influence is a more reliable criterion than activation magnitude for identifying features that causally mediate refusal behavior.

Related papers

• Detecting Contaminated Code-Generation Prompt Batches via Influence Functions [4.185678330063222]
  Large language models (LLMs) are increasingly used for code generation, yet they remain vulnerable to prompts that elicit insecure implementations.<n>We propose CodeSIFT, a threat-model-agnostic detection method that leverages influence functions to identify batches of prompts that induce anomalous model behavior.
  arXiv  Detail & Related papers  (2026-08-14T13:37:53Z)
• Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning [63.24624171679711]
  Large language model (LLM)-based agents often make suboptimal tool-use decisions.<n>We propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation.
  arXiv  Detail & Related papers  (2026-06-05T07:08:34Z)
• Uncertainty-Aware Jamming Mitigation with Active RIS: A Robust Stackelberg Game Approach [65.06640919319413]
  This paper investigates the jamming mitigation by leveraging an active reconfigurable intelligent surface (ARIS)<n>We adopt the Stackelberg game formulation to model the strategic interaction between the legitimate side and the adversary.<n>We first derive the optimal jamming policy as the follower's best response, which is then incorporated into the legitimate-side optimization for robust anti-jamming design.
  arXiv  Detail & Related papers  (2026-02-20T12:02:01Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics [2.4839105527363574]
  We introduce Refusal Steering, an inference-time method to exercise fine-grained control over Large Language Models refusal behaviour.<n>We show that it can remove political refusal behaviour while retaining safety alignment for harmful content.
  arXiv  Detail & Related papers  (2025-12-18T14:43:04Z)
• Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning? [68.82210578851442]
  We investigate why safety alignment fails in reasoning models through a mechanistic interpretability lens.<n>Using a linear probing approach to trace refusal intentions across token positions, we discover a phenomenon termed as textbfrefusal cliff<n>We propose textbfCliff-as-a-Judge, a novel data selection method that identifies training examples exhibiting the largest refusal cliff to efficiently repair reasoning models' safety alignment.
  arXiv  Detail & Related papers  (2025-10-07T15:32:59Z)
• Understanding Refusal in Language Models with Sparse Autoencoders [27.212781538459588]
  We use sparse autoencoders to identify latent features that causally mediate refusal behaviors.<n>We intervene on refusal-related features to assess their influence on generation.<n>This enables a fine-grained inspection of how refusal manifests at the activation level.
  arXiv  Detail & Related papers  (2025-05-29T15:33:39Z)
• Defending the Edge: Representative-Attention for Mitigating Backdoor Attacks in Federated Learning [7.808916974942399]
  heterogeneous edge devices produce diverse, non-independent, and identically distributed (non-IID) data.<n>We propose a novel representative-attention-based defense mechanism, named FeRA, to distinguish benign from malicious clients.<n>Our evaluation demonstrates FeRA's robustness across various FL scenarios, including challenging non-IID data distributions typical of edge devices.
  arXiv  Detail & Related papers  (2025-05-15T13:44:32Z)
• Feature-Aware Malicious Output Detection and Mitigation [8.378272216429954]
  We propose a feature-aware method for harmful response rejection (FMM)<n>FMM detects the presence of malicious features within the model's feature space and adaptively adjusts the model's rejection mechanism.<n> Experimental results demonstrate the effectiveness of our approach across multiple language models and diverse attack techniques.
  arXiv  Detail & Related papers  (2025-04-12T12:12:51Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications.<n>Our research identifies two critical latent factors affecting RAG's confidence in its predictions.<n>We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Causal Disentangled Variational Auto-Encoder for Preference Understanding in Recommendation [50.93536377097659]
  This paper introduces the Causal Disentangled Variational Auto-Encoder (CaD-VAE), a novel approach for learning causal disentangled representations from interaction data in recommender systems. The approach utilizes structural causal models to generate causal representations that describe the causal relationship between latent factors.
  arXiv  Detail & Related papers  (2023-04-17T00:10:56Z)
• ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning [2.984934409689467]
  Causal confusion refers to a phenomenon where an agent learns spurious correlations between features which might not hold across the entire state space. We propose ReCCoVER, an algorithm which detects causal confusion in agent's reasoning before deployment.
  arXiv  Detail & Related papers  (2022-03-21T13:17:30Z)
• DARTS-: Robustly Stepping out of Performance Collapse Without Indicators [74.21019737169675]
  Differentiable architecture search suffers from long-standing performance instability. indicators such as Hessian eigenvalues are proposed as a signal to stop searching before the performance collapses. In this paper, we undertake a more subtle and direct approach to resolve the collapse.
  arXiv  Detail & Related papers  (2020-09-02T12:54:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.