The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind

• URL: http://arxiv.org/abs/2509.20393v1
• Date: Tue, 23 Sep 2025 04:52:40 GMT
• Title: The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind
• Authors: Caleb DeLeeuw, Gaurav Chawla, Aniket Sharma, Vanessa Dietze,
• Abstract summary: Secret Agenda reliably induced lying when deception advantaged goal achievement across all model families.<n>Analysis revealed that autolabeled SAE features for "deception" rarely activated during strategic dishonesty.<n>Findings suggest autolabel-driven interpretability approaches fail to detect or control behavioral deception.
• Score: 0.23332469289621785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate strategic deception in large language models using two complementary testbeds: Secret Agenda (across 38 models) and Insider Trading compliance (via SAE architectures). Secret Agenda reliably induced lying when deception advantaged goal achievement across all model families. Analysis revealed that autolabeled SAE features for "deception" rarely activated during strategic dishonesty, and feature steering experiments across 100+ deception-related features failed to prevent lying. Conversely, insider trading analysis using unlabeled SAE activations separated deceptive versus compliant responses through discriminative patterns in heatmaps and t-SNE visualizations. These findings suggest autolabel-driven interpretability approaches fail to detect or control behavioral deception, while aggregate unlabeled activations provide population-level structure for risk assessment. Results span Llama 8B/70B SAE implementations and GemmaScope under resource constraints, representing preliminary findings that motivate larger studies on feature discovery, labeling methodology, and causal interventions in realistic deception contexts.

Related papers

• Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection [4.514361164656055]
  We introduce a taxonomy of ten categories of hidden intentions, organised by intent, mechanism, context, and impact.<n>We systematically assess detection methods, including reasoning and non-reasoning LLM judges.<n>We find that detection collapses in realistic open-world settings, particularly under low-prevalence conditions.
  arXiv  Detail & Related papers  (2026-01-26T14:59:17Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Measure-Theoretic Anti-Causal Representation Learning [29.12751904333385]
  Causal representation learning in the anti-causal setting (labels cause features rather than the reverse) presents unique challenges.<n>We propose Anti-Causal Invariant Abstractions (ACIA), a novel measure-theoretic framework for anti-causal representation learning.<n>ACIA employs a two-level design, low-level representations capture how labels generate observations, while high-level representations learn stable causal patterns across environment-specific variations.
  arXiv  Detail & Related papers  (2025-10-16T22:13:05Z)
• Preliminary Investigation into Uncertainty-Aware Attack Stage Classification [81.28215542218724]
  This work addresses the problem of attack stage inference under uncertainty.<n>We propose a classification approach based on Evidential Deep Learning (EDL), which models predictive uncertainty by outputting parameters of a Dirichlet distribution over possible stages.<n>Preliminary experiments in a simulated environment demonstrate that the proposed model can accurately infer the stage of an attack with confidence.
  arXiv  Detail & Related papers  (2025-08-01T06:58:00Z)
• DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective [70.77570343385928]
  We introduce a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing)<n>We formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset.<n>Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery.<n>Our benchmark, DATABench, comprises 17 evasion attacks, 5 forgery attacks, and 9
  arXiv  Detail & Related papers  (2025-07-08T03:07:15Z)
• Addressing Key Challenges of Adversarial Attacks and Defenses in the Tabular Domain: A Methodological Framework for Coherence and Consistency [25.830427564563422]
  Class-Specific Anomaly Detection (CSAD) is an effective novel anomaly detection approach.<n> CSAD evaluates adversarial samples relative to their predicted class distribution, rather than a broad benign distribution.<n>Our evaluation incorporates both anomaly detection rates with SHAP-based assessments to provide a more comprehensive measure of adversarial sample quality.
  arXiv  Detail & Related papers  (2024-12-10T09:17:09Z)
• Hide in Plain Sight: Clean-Label Backdoor for Auditing Membership Inference [16.893873979953593]
  We propose a novel clean-label backdoor-based approach for stealthy data auditing. Our approach employs an optimal trigger generated by a shadow model that mimics target model's behavior. The proposed method enables robust data auditing through blackbox access, achieving high attack success rates across diverse datasets.
  arXiv  Detail & Related papers  (2024-11-24T20:56:18Z)
• Understanding the Effectiveness of Coverage Criteria for Large Language Models: A Special Angle from Jailbreak Attacks [10.909463767558023]
  Large language models (LLMs) have revolutionized artificial intelligence, but their deployment across critical domains has raised concerns about their abnormal behaviors when faced with malicious attacks.<n>In this paper, we conduct a comprehensive empirical study to evaluate the effectiveness of traditional coverage criteria in identifying such inadequacies.<n>We develop a real-time jailbreak detection mechanism that achieves high accuracy (93.61% on average) in classifying queries as normal or jailbreak.
  arXiv  Detail & Related papers  (2024-08-27T17:14:21Z)
• Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning [64.78972193105443]
  This paper presents a novel AE detection framework, named trustworthy for predictions. performs the detection by distinguishing the AE's abnormal relation with its augmented versions. An off-the-shelf Self-Supervised Learning (SSL) model is used to extract the representation and predict the label.
  arXiv  Detail & Related papers  (2022-08-31T08:18:44Z)
• SAGE: Intrusion Alert-driven Attack Graph Extractor [4.530678016396476]
  Attack graphs (AGs) are used to assess pathways availed by cyber adversaries to penetrate a network. We propose to automatically learn AGs based on actions observed through intrusion alerts, without prior expert knowledge.
  arXiv  Detail & Related papers  (2021-07-06T17:45:02Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)
• Foreseeing the Benefits of Incidental Supervision [83.08441990812636]
  This paper studies whether we can, in a single framework, quantify the benefits of various types of incidental signals for a given target task without going through experiments. We propose a unified PAC-Bayesian motivated informativeness measure, PABI, that characterizes the uncertainty reduction provided by incidental supervision signals.
  arXiv  Detail & Related papers  (2020-06-09T20:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.