Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations

• URL: http://arxiv.org/abs/2605.27958v1
• Date: Wed, 27 May 2026 04:51:55 GMT
• Title: Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
• Abstract summary: Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift.<n>This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters)<n>We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy.
• Score: 5.218766876318545
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift. This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters), diagnosing why they fail rather than merely documenting that they fail. We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy. Our design includes cross-domain transfer matrices, multi-dimensional probe analysis with permutation null baselines, entropy-residualization tests, and distractor evaluations across 8 stylistic shifts. We find that: (a) probes achieve near-perfect AUROC (>=0.998) on clean data but collapse under stylistic shifts; style-augmented probes recover near-perfect detection (mean AUROC 0.979-0.983) on unseen styles; (b) the single-direction hypothesis is rejected (k=1 captures only 0.61-0.80 AUROC), with cross-domain transfer failure confirmed as geometric rather than layer-mismatch-driven; (c) the entropy-proxy hypothesis is rejected (max |rho|=0.454, max Delta-AUROC after residualization=0.004); and (d) deception does not form a significant linear subspace (per-domain k*=0), yet multi-dimensional probes (k>=5) recover the signal through distributed sub-threshold features. Probe fragility reflects distributional narrowness rather than an architectural limitation: style-augmented probes recover near-perfect detection at both 4B and 27B, establishing that the inverse scaling pattern is a training-distribution artifact rather than a genuine scale-dependent phenomenon.

Related papers

• Exact Degeneracy Under Balanced k-Shot Sampling:Consequences for Small-Sample Discriminant Analysis on LLM Embeddings [0.0]
  We show that balanced k-shot sampling induces an exact, provable degeneracy in a family of small-sample discriminant estimators.<n>We then evaluate the Kernelgression repaired framework on few-shot text classification on frozen embeddings.
  arXiv  Detail & Related papers  (2026-09-09T08:13:54Z)
• Curved Inference II: Sleeper Agent Geometry - Extending Interpretability Beyond Probes [0.0]
  This paper extends Anthropic's Sleeper Agents research, which showed artificial backdoors persist through safety training & can be detected by linear probes with >99% accuracy.<n>We introduce a naturalistic methodology using multi-turn context windows that simulates realistic deceptive reasoning without artificial triggers or supervised backdoor insertion.<n> Geometric structure reliably predicts semantic classification, with statistically significant differences in surface area across five prompt strategies & two model families.
  arXiv  Detail & Related papers  (2026-08-25T03:51:20Z)
• GLID: Gated Local Intrinsic Dimension Repairs the Blind Spots of Face-Forgery Detectors [4.564848621539382]
  Fine-tuned foundation-model detectors dominate face-forgery benchmarks, yet they stay blind to generator families absent from training.<n>We present GLID, a detector that repairs this blind spot with geometry instead of data.<n>GLID treats the patch tokens of a single image as a sample from a manifold and estimates their local intrinsic dimension (LID) at several depths of a frozen vision transformer.
  arXiv  Detail & Related papers  (2026-07-21T06:52:46Z)
• Need We Teach Foundation Models What is a Generative Image? Gradient-Free Generative Artifact Detection via Analytic Spectral Adaptation [3.881211374324378]
  Adapting foundation models to detect generative artifacts via gradient-based updates compromises their intrinsic representations.<n>We propose a gradient-free methodology reframing detection from binary classification to an out-of-distribution (OOD) anomaly measurement problem.
  arXiv  Detail & Related papers  (2026-06-03T13:04:23Z)
• Reading Calibrated Uncertainty from Language Model Trajectories [46.663987199083245]
  Methods that probe the model's internal activations feed raw hidden states into opaque snapshots and leave implicit the layer-wise trajectory by which a representation is formed.<n>We extract eleven scale-invariant geometric features, tracing the cumulative path of per-layer updates, and feed them to a sparse linear probe.<n>The probe outperforms MSP under selective abstention, with baseline scaling with up to 21 AURC points.
  arXiv  Detail & Related papers  (2026-05-19T19:24:29Z)
• Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams [0.0]
  Harmful intent is geometrically recoverable from large language model residual streams.<n>We characterise this geometry through six direction-finding strategies.<n>A direction fitted on AdvBench transfers to held-out HarmBench and JailbreakBench with worst-case AUROC 0.96.
  arXiv  Detail & Related papers  (2026-04-20T23:02:37Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Topological Residual Asymmetry for Bivariate Causal Direction [1.2691047660244335]
  Topological Residual Asymmetry is a geometry-based criterion for additive-noise models.<n>We quantify a bulk-tube contrast using a 0D persistent-homology functional.<n>Experiments across many challenging synthetic and real-data scenarios demonstrate the method's superiority.
  arXiv  Detail & Related papers  (2026-01-31T00:38:53Z)
• ARGUS: Adaptive Rotation-Invariant Geometric Unsupervised System [0.0]
  This paper introduces Argus, a framework that reconceptualizes drift detection as tracking local statistics over a fixed spatial partition of the data manifold.<n> Voronoi tessellations over canonical orthonormal frames yield drift metrics that are invariant to transformations.<n>A graph-theoretic characterization of drift propagation is developed that distinguishes coherent distributional shifts from isolated perturbations.
  arXiv  Detail & Related papers  (2026-01-03T22:39:20Z)
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection [2.8547732086436306]
  A fundamental limitation of supervised deep learning is "Generalization Collapse"<n>We propose Latent Sculpting, a hierarchical two-stage representation learning framework.<n>We report an 88.89% detection rate on "Infiltration" scenarios.
  arXiv  Detail & Related papers  (2025-12-19T11:37:02Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• Orthogonal Matrix Retrieval with Spatial Consensus for 3D Unknown-View Tomography [58.60249163402822]
  Unknown-view tomography (UVT) reconstructs a 3D density map from its 2D projections at unknown, random orientations. The proposed OMR is more robust and performs significantly better than the previous state-of-the-art OMR approach.
  arXiv  Detail & Related papers  (2022-07-06T21:40:59Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.