Transcoders for Investigating Deception in Language Models

• URL: http://arxiv.org/abs/2607.14791v1
• Date: Thu, 16 Jul 2026 10:10:10 GMT
• Title: Transcoders for Investigating Deception in Language Models
• Abstract summary: We investigate the use of transcoders to analyse deceptive behaviour in language models.<n>We identify a dictionary of deception-related features that exert a stronger influence on deceptive outputs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transcoders have recently emerged as a promising approach for mechanistic interpretability (MI), enabling circuit-level analysis of model behaviour. In this paper, we investigate the use of transcoders to analyse deceptive behaviour in language models, a behaviour that poses a safety and security risk. Using a Qwen3-4B model with pre-trained transcoders, specifically per-layer transcoders (PLTs), we construct attribution graphs that capture feature activations and inter-feature dependencies, allowing circuit-level analysis of deception. Through feature steering and circuit analysis, we identified a dictionary of deception-related features and show that these features exert a stronger influence on deceptive outputs, as they produce predictable shifts between deceptive and non-deceptive responses. These findings suggest that deception emerges from internal model mechanisms and highlight the potential of transcoders for behavioural monitoring and early detection of security vulnerabilities related to malicious behaviours in language models.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning [0.0]
  mechanistic interpretability is an emerging field that seeks to reverse-engineer the internal algorithms of modern neural networks.<n>The paper provides a detailed examination of Transformer circuit analysis, exploring how internal components drive complex tasks and in-context learning.<n>It then tackles the core challenge of superposition and polysemanticity, demonstrating how tools like Sparse Autoencoders (SAEs) and transcoders can decompose tangled network activations into distinct, human-interpretable features.
  arXiv  Detail & Related papers  (2026-07-08T12:04:38Z)
• Explainability-Guided Adversarial Attacks on Transformer-Based Malware Detectors Using Control Flow Graphs [0.19116784879310025]
  This paper examines the vulnerability of a RoBERTa-based malware detector that linearizes control flow graphs into sequences of function calls.<n>By evaluating evasion strategies within this graph-to-sequence framework, we provide insight into the practical robustness of transformer-based malware detectors beyond aggregate detection accuracy.
  arXiv  Detail & Related papers  (2026-04-04T19:50:04Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Decoding the Critique Mechanism in Large Reasoning Models [50.821607345799386]
  Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions.<n>We investigate how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps.<n>We find that despite the error propagating through the chain-of-thought, the model still reaches the correct final answer.
  arXiv  Detail & Related papers  (2026-03-17T10:03:30Z)
• Backdoor Directions in Vision Transformers [56.382912038371046]
  This paper investigates how Backdoor Attacks are represented within Vision Transformers (ViTs)<n>We identify a specific trigger direction'' in the model's activations that corresponds to the internal representation of the trigger.<n>Using this direction as a diagnostic tool, we trace how backdoor features are processed across layers.
  arXiv  Detail & Related papers  (2026-03-11T14:13:48Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Feature-Aware Test Generation for Deep Learning Models [0.5368630420272898]
  We introduce Detect, a feature-aware test generation framework for vision-based deep learning (DL) models.<n>It generates inputs by perturbing disentangled semantic attributes within the latent space.<n>It identifies which features lead to behavior shifts and uses a vision-language model for semantic attribution.
  arXiv  Detail & Related papers  (2026-01-20T15:41:06Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• Interpretable Anomaly Detection in Encrypted Traffic Using SHAP with Machine Learning Models [0.0]
  This study aims to develop an interpretable machine learning-based framework for anomaly detection in encrypted network traffic.<n>Models are trained and evaluated on three benchmark encrypted traffic datasets.<n> SHAP visualizations successfully revealed the most influential traffic features contributing to anomaly predictions.
  arXiv  Detail & Related papers  (2025-05-22T05:50:39Z)
• Automated Static Warning Identification via Path-based Semantic Representation [37.70518599085676]
  This paper employs deep neural networks' powerful feature extraction and representation abilities to generate code semantics from control flow graph paths for warning identification. We fine-tune the pre-trained language model to encode the path sequences and capture the semantic representations for model building.
  arXiv  Detail & Related papers  (2023-06-27T15:46:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.