Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention

• URL: http://arxiv.org/abs/2606.11205v1
• Date: Wed, 22 Apr 2026 13:49:34 GMT
• Title: Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention
• Abstract summary: We introduce dual-stance evaluation, which tests both stances of each topic, and apply it to centroid-difference steering on Llama-3-8B-Instruct.<n>We find a dissociation: the model represents sycophantic and factual agreement in geometrically distinct subspaces, yet the steering direction projects equally onto both and cannot differentially target either.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation steering can shift LLM behaviour, but standard evaluations do not typically test whether a sycophancy-reduction direction also suppresses agreement with factually correct statements. We introduce dual-stance evaluation, which tests both stances of each topic, and apply it to centroid-difference steering on Llama-3-8B-Instruct. We find a dissociation: the model represents sycophantic and factual agreement in geometrically distinct subspaces, yet the steering direction projects equally onto both and cannot differentially target either. The direction accordingly reduces agreement with factually correct statements (e.g. that the Earth is round) as well as sycophantic ones. All other static properties of the two activation groups are matched, suggesting the behavioural dissociation arises from generation dynamics or from finer-grained structure that residual-stream analysis cannot resolve. The pattern illustrates a general gap: representations that are readable from activations may not be writable through them.

Related papers

• THESIS-MoE: Trainable Hierarchical Extraction and SteerIng of Sycophancy in Mixture-of-Experts [0.300988853836121]
  Sycophancy, the tendency of a language model to change its answer to match a user's stated belief, is a common alignment failure.<n>We introduce a shared contrastive signal, built from matched prompts with and without a stated belief, that identifies where sycophancy lives.<n>Our results demonstrate that sycophancy resides in identifiable computational subcircuits and can be selectively steered.
  arXiv  Detail & Related papers  (2026-08-16T11:30:26Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention [48.20065816234311]
  Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they struggle with the vision-override phenomenon.<n>We propose CofactVLA, a novel causal intervention framework.<n>We show that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks.
  arXiv  Detail & Related papers  (2026-08-05T02:58:41Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• R-APS: Compositional Reasoning and In-Context Meta-Learning for Constrained Design via Reflective Adversarial Pareto Search [18.8323743697237]
  Large language models (LLMs) are fluent on open-ended tasks, yet in agentic settings, fluency does not ensure reliable delivery.<n>We trace this gap to three coupled structural failures: errors propagate without localization, worst-case perturbations go unevaluated, and accumulated knowledge is never invalidated.<n>We argue these share a root cause: abductive, counterfactual, meta-inductive, corrective, and inductive reasoning pull a shared context in incompatible directions.
  arXiv  Detail & Related papers  (2026-06-03T12:45:39Z)
• Departure from Regularity: Degree Heterogeneity and Eigengap as the Structural Drivers of ASE-LSE Latent Subspace Disagreement [2.2201528765499416]
  Two methods for analysing graph data, Adjacency Spectral Embedding and Laplacian Spectral Embedding, often produce different results when applied to the same network.<n>We show that regularity is a sufficient condition for perfect agreement.<n>We prove an explicit bound whose two terms suggest the structural ingredients controlling it: degree heterogeneity, which pushes the methods apart, and community structure strength, which pulls them back together.
  arXiv  Detail & Related papers  (2026-05-21T11:30:37Z)
• The Cylindrical Representation Hypothesis for Language Model Steering [57.97381760521523]
  We show that a central axis captures the main difference between concept absence and presence and drives concept generation.<n>We formalize this as the Cylindrical Representation Hypothesis (CRH)<n>Our experiments verify the existence of the cylindrical structure and demonstrate that CRH provides a valid and practical way to interpret model steering behavior in real settings.
  arXiv  Detail & Related papers  (2026-05-03T12:26:13Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment [49.68063561145927]
  We propose a unified ordinary differential equations (ODEs)-based theoretical framework for activation steering.<n>We introduce ODESteer, a kind of ODE-based steering guided by barrier functions.<n>Compared to state-of-the-art activation steering methods, ODESteer achieves consistent empirical improvements.
  arXiv  Detail & Related papers  (2026-02-19T17:13:44Z)
• Fundamental limits to contrast reversal of self-fidelity correlations [0.0]
  In measurement design, it is common to engineer anti-contrast readouts.<n>We ask whether two evolutions can be made uniformly opposite over a broad input ensemble.<n>We adopt the Pearson correlation coefficient as a device-agnostic measure of global opposition between two evolutions.
  arXiv  Detail & Related papers  (2025-09-30T15:07:28Z)
• Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs [1.9693252204587723]
  We decompose sycophantic agreement and sycophantic praise, contrasting both with genuine agreement.<n>Results suggest that sycophantic behaviors correspond to distinct, independently steerable representations.
  arXiv  Detail & Related papers  (2025-09-25T15:19:39Z)
• Nested Counterfactual Identification from Arbitrary Surrogate Experiments [95.48089725859298]
  We study the identification of nested counterfactuals from an arbitrary combination of observations and experiments. Specifically, we prove the counterfactual unnesting theorem (CUT), which allows one to map arbitrary nested counterfactuals to unnested ones.
  arXiv  Detail & Related papers  (2021-07-07T12:51:04Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)
• GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement [91.9003001845855]
  VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. We address VAE-based unsupervised disentanglement by leveraging the constraints derived from the Group Theory based definition as the non-probabilistic inductive bias. We train 1800 models covering the most prominent VAE-based models on five datasets to verify the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-02-20T09:49:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.