Probabilistic Concept-Aware Steering for Trustworthy LLM Inference

• URL: http://arxiv.org/abs/2607.18259v1
• Date: Fri, 15 May 2026 15:34:48 GMT
• Title: Probabilistic Concept-Aware Steering for Trustworthy LLM Inference
• Abstract summary: We present the Probabilistic Concept-Aware Steering (PCS) framework for large language models (LLMs) inference.<n>PCS preserves original task competence while providing controllable, safety-oriented semantic bias through concept-driven steering-vector retrieval and probabilistic strength calibration.
• Score: 14.752999429320226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Steering vectors (SVs), an inference-time intervention technique for large language models (LLMs), guide the generation process by adding a concept-specific direction vector to intermediate activations during inference. However, existing SV methods frequently yield representation-incoherent behaviors that undermine interpretability and fine-grained control, largely because prior work has focused on binary positive-negative steering evaluation while employing discrete clustering metrics that fail to capture the continuous spectrum of semantic alignment. In this work, we present the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference. PCS preserves original task competence while providing controllable, safety-oriented semantic bias through concept-driven steering-vector retrieval and probabilistic strength calibration.

Related papers

• Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving [37.533498955258544]
  We propose a single-pass guidance mechanism that replaces instance-level residuals with class-level latent shifts.<n>We demonstrate that LCS reduces inference latency by approximately 50% while achieving stronger command adherence.
  arXiv  Detail & Related papers  (2026-07-31T19:32:28Z)
• Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention [33.40772052822545]
  This paper bridges the gap between mechanistic interpretability and actionable control.<n>We first present a systematic analysis using structural, causal, and geometric probes.<n>We then operationalize these interpretability insights into a suite of training-free, decode-time interventions.
  arXiv  Detail & Related papers  (2026-05-31T13:52:24Z)
• Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence [6.28866496877782]
  In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern.<n>We introduce a concept of self-function vectors, built upon Bayesian views and the mechanistic interpretability of ICL.<n>These vectors leverage internal model representations to model the latent concept learned during in-context prompting.<n>We show that our proposed methodology can measure uncertainty of LLM predictions more reliably than existing alternative methods.
  arXiv  Detail & Related papers  (2026-04-28T09:47:40Z)
• FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering [11.609466767597093]
  FaithSteer-BENCH is a stress-testing benchmark forference-time steering.<n>It evaluates steering methods at a fixed deployment-style operating point through three gate-wise criteria: controllability, utility preservation, and robustness.<n> Gate-wise benchmark results show that existing methods do not necessarily provide reliable controllability in deployment-oriented practical settings.
  arXiv  Detail & Related papers  (2026-03-18T22:28:36Z)
• Mechanistic Indicators of Steering Effectiveness in Large Language Models [3.635648354808971]
  Activation-based steering enables Large Language Models to exhibit targeted behaviors by intervening on intermediate activations without retraining.<n>Despite its widespread use, the mechanistic factors that govern when steering succeeds or fails remain poorly understood.<n>We investigate whether the reliability of steering can be diagnosed using internal model signals.
  arXiv  Detail & Related papers  (2026-02-02T06:56:22Z)
• Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models [12.506018278890862]
  We propose a differentiable concept scoring function whose local gradient defines the steering direction at each activation.<n>This formulation supports coordinated multi-layer interventions in a shared, aligned concept space.<n>Across multiple language models, SVF delivers stronger and more reliable control, improving the practicality of inference-time steering.
  arXiv  Detail & Related papers  (2026-02-02T05:14:42Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learning [55.83415345423854]
  We propose COVLM-RL, a novel end-to-end driving framework that integrates Critical Object-oriented (CO) reasoning with reinforcement learning (RL)<n>Experiments conducted in the CARLA simulator demonstrate that COVLM-RL significantly improves the success rate by 30% in trained driving environments.
  arXiv  Detail & Related papers  (2025-12-10T06:18:16Z)
• Latent Chain-of-Thought for Visual Reasoning [53.541579327424046]
  Chain-of-thought (CoT) reasoning is critical for improving the interpretability and reliability of Large Vision-Language Models (LVLMs)<n>We reformulate reasoning in LVLMs as posterior inference and propose a scalable training algorithm based on amortized variational inference.<n>We empirically demonstrate that the proposed method enhances the state-of-the-art LVLMs on seven reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-27T23:10:06Z)
• Causal Unsupervised Semantic Segmentation [60.178274138753174]
  Unsupervised semantic segmentation aims to achieve high-quality semantic grouping without human-labeled annotations. We propose a novel framework, CAusal Unsupervised Semantic sEgmentation (CAUSE), which leverages insights from causal inference.
  arXiv  Detail & Related papers  (2023-10-11T10:54:44Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)
• Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [77.39239190539871]
  We show how uncertainty estimation can be leveraged to enable safety critical image segmentation in autonomous driving. We introduce a new uncertainty measure based on disagreeing predictions as measured by a dissimilarity function. We show experimentally that our proposed approach is much less computationally intensive at inference time than competing methods.
  arXiv  Detail & Related papers  (2021-05-28T09:23:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.