PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancy

• URL: http://arxiv.org/abs/2608.16650v1
• Date: Mon, 17 Aug 2026 14:48:49 GMT
• Title: PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancy
• Abstract summary: Large language models (LLMs) exhibit sycophancy, a tendency to agree with user beliefs regardless of factual accuracy.<n>We introduce PCA-guided Activation Scaling (PAS), an activation steering framework that decomposes residual stream activations into a PCA-identified sycophancy-honesty subspace.<n>Across three LLMs and three datasets, PAS achieves strong monotonicity (Spearman $$ = +0.92) and an average shift of 15.4% per direction.
• Score: 15.914340276284085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) exhibit sycophancy, a tendency to agree with user beliefs regardless of factual accuracy. This can reinforce misconceptions, but eliminating it entirely risks over-correction against valid opinions. Effective control must therefore both reduce and increase sycophancy with predictable and gradual effect. Yet, existing methods fail to ensure a bidirectional and monotonic relationship between steering strength and behavioral outcome across models and datasets. We introduce PCA-guided Activation Scaling (PAS), an activation steering framework that decomposes residual stream activations into a PCA-identified sycophancy-honesty subspace and an orthogonal residual, then applies distinct scaling exponents to achieve monotonic, bidirectional control. Across three LLMs and three datasets, PAS achieves strong monotonicity (Spearman $ρ$ = +0.92) and an average shift of 15.4% per direction, compared with 8.7% for the baselines. Ablation studies confirm that the decomposition, asymmetric exponents, and layer selection are each essential for maintaining monotonic control. The data and code are available at https://github.com/Bellafc/PCS.

Related papers

• CRAFT: Counterfactual Credit Assignment from Free Sibling Rollouts for Self-Distilled Agentic Reinforcement Learning [4.663685189987781]
  Self-distilled agentic reinforcement learning augments trajectory-level reward with a token-level distillation loss.<n>The prevailing recipe gates this loss by a single scalar, the teacher-student log-probability gap.<n>We introduce CRAFT, a threepillar credit-assignment scheme that addresses both limitations.
  arXiv  Detail & Related papers  (2026-06-28T16:11:47Z)
• GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs [5.018957704056544]
  Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.<n>We show that when multiple semantic directions are superposed without constraints, the model collapses.<n>We propose GEMS, a training-free method that maps each source to a corresponding geometric constraint.
  arXiv  Detail & Related papers  (2026-06-18T08:43:33Z)
• DISA: Offline Importance Sampling for Distribution-Matching LLM-RL [56.9445657766829]
  We introduce DISA, which moves this calibration problem outside the RL loop.<n> DISA draws proposal trajectories offline, estimates the partition function via importance sampling, and freezes the resulting partition-function estimate.<n>On two open-weight backbones across six math and three code benchmarks, DISA matches or exceeds the online-coupled distribution-matching baseline Flow.
  arXiv  Detail & Related papers  (2026-05-17T07:14:44Z)
• Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence [16.403654360036498]
  misalignment can be triggered by adversarial prompts, benign fine-tuning, emergent misalignment, and goal misgeneralization.<n>Recent evidence suggests that some misalignment behaviors are encoded as linear structure in activation space, making it tractable via steering.<n>These findings motivate activation steering as a lightweight runtime defense that continuously corrects misaligned activations throughout generation.
  arXiv  Detail & Related papers  (2026-04-09T12:28:22Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• ROAST: Rollout-based On-distribution Activation Steering Technique [16.632201561391366]
  Activation steering provides parameter-efficient control over large language models at inference time.<n>We propose ROAST (Rollout-based On-distribution Activation Steering Technique), which estimates steering directions from the model's own on-distribution rollouts via ROC.<n>Our empirical analysis reveals that while activation magnitude correlates moderately with directional consistency, the variance in magnitude is significant and often disproportionate to semantic quality.
  arXiv  Detail & Related papers  (2026-02-15T13:30:26Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• Backdoor Cleaning without External Guidance in MLLM Fine-tuning [76.82121084745785]
  Believe Your Eyes (BYE) is a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples.<n>It achieves near-zero attack success rates while maintaining clean-task performance.
  arXiv  Detail & Related papers  (2025-05-22T17:11:58Z)
• TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets [118.22975463000928]
  We consider an offline reinforcement learning (RL) setting where the agent need to learn from a dataset collected by rolling out multiple behavior policies. There are two challenges for this setting: 1) The optimal trade-off between optimizing the RL signal and the behavior cloning (BC) signal changes on different states due to the variation of the action coverage induced by different behavior policies. In this paper, we address both challenges by using adaptively weighted reverse Kullback-Leibler (KL) divergence as the BC regularizer based on the TD3 algorithm.
  arXiv  Detail & Related papers  (2022-12-05T09:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.