Concept-Level Explainability for Auditing & Steering LLM Responses

• URL: http://arxiv.org/abs/2505.07610v2
• Date: Mon, 19 May 2025 14:00:52 GMT
• Title: Concept-Level Explainability for Auditing & Steering LLM Responses
• Abstract summary: ConceptX is a model-agnostic, concept-level explainability method.<n>It identifies semantically rich tokens in the prompt, and assigns them importance based on the outputs' semantic similarity.<n>It enables both auditing, by uncovering sources of bias, and steering, by modifying prompts to shift the sentiment or reduce the harmfulness of LLM responses.
• Score: 12.089513278445704
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: As large language models (LLMs) become widely deployed, concerns about their safety and alignment grow. An approach to steer LLM behavior, such as mitigating biases or defending against jailbreaks, is to identify which parts of a prompt influence specific aspects of the model's output. Token-level attribution methods offer a promising solution, but still struggle in text generation, explaining the presence of each token in the output separately, rather than the underlying semantics of the entire LLM response. We introduce ConceptX, a model-agnostic, concept-level explainability method that identifies the concepts, i.e., semantically rich tokens in the prompt, and assigns them importance based on the outputs' semantic similarity. Unlike current token-level methods, ConceptX also offers to preserve context integrity through in-place token replacements and supports flexible explanation goals, e.g., gender bias. ConceptX enables both auditing, by uncovering sources of bias, and steering, by modifying prompts to shift the sentiment or reduce the harmfulness of LLM responses, without requiring retraining. Across three LLMs, ConceptX outperforms token-level methods like TokenSHAP in both faithfulness and human alignment. Steering tasks boost sentiment shift by 0.252 versus 0.131 for random edits and lower attack success rates from 0.463 to 0.242, outperforming attribution and paraphrasing baselines. While prompt engineering and self-explaining methods sometimes yield safer responses, ConceptX offers a transparent and faithful alternative for improving LLM safety and alignment, demonstrating the practical value of attribution-based explainability in guiding LLM behavior.

Related papers

• Correcting Influence: Unboxing LLM Outputs with Orthogonal Latent Spaces [9.24259308424615]
  We introduce a flexible framework that infers token-level influence through a latent mediation approach for general prediction tasks.<n>Unlike prior methods where influence decomposes additively across tokens, influence computed over latent features is inherently non-decomposable.<n>Our framework enhances trust and enables model auditing, generalizing to high-stakes domain requiring transparent and accountable decisions.
  arXiv  Detail & Related papers  (2026-05-12T23:01:29Z)
• Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance [57.17102098930037]
  Large language models (LLMs) achieve strong performance and have revolutionized NLP.<n>Their lack of explainability keeps them treated as black boxes, limiting their use in domains that demand transparency and trust.<n>We introduce a training-free method that enhances faithfulness by guiding explanation generation through attention-level interventions.
  arXiv  Detail & Related papers  (2026-04-15T18:32:32Z)
• Inside Out: Uncovering How Comment Internalization Steers LLMs for Better or Worse [4.911314444215628]
  This study is the first concept-level interpretability study of Large Language Models (LLM) in Software Engineering.<n>We analyze three tasks - code completion, translation, and refinement - through the lens of internal comment representation.<n>We find that code summarization consistently triggered the strongest activation of comment concepts, whereas code completion elicited the weakest sensitivity.
  arXiv  Detail & Related papers  (2025-12-18T17:24:56Z)
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• From Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generation [3.75886080255807]
  We introduce two innovative attack frameworks designed to generate dynamic and adaptive adversarial examples.<n>We produce subtle and natural-looking adversarial inputs that preserve semantic similarity to the original text.<n>Our attacks evolve with the advancements in LLMs and demonstrate strong transferability acrossversa unknown to the attacker.
  arXiv  Detail & Related papers  (2025-11-05T02:27:56Z)
• Does Representation Intervention Really Identify Desired Concepts and Elicit Alignment? [73.80382983108997]
  Representation intervention aims to locate and modify the representations that encode the underlying concepts in Large Language Models.<n>If the interventions are faithful, the intervened LLMs should erase the harmful concepts and be robust to both in-distribution adversarial prompts and the out-of-distribution jailbreaks.<n>We propose Concept Concentration (COCA), which simplifies the decision boundary between harmful and benign representations.
  arXiv  Detail & Related papers  (2025-05-24T12:23:52Z)
• Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding [48.52389201779425]
  Speculative decoding accelerates inference by generating multiple draft tokens using a lightweight model and verifying them in parallel.<n>Existing verification methods rely heavily on distributional consistency while overlooking semantic correctness.<n>We propose Reflective Verification, a training-free and semantics-aware approach that achieves a better trade-off between correctness and efficiency.
  arXiv  Detail & Related papers  (2025-05-24T10:26:27Z)
• Improving LLM Safety Alignment with Dual-Objective Optimization [65.41451412400609]
  Existing training-time safety alignment techniques for large language models (LLMs) remain vulnerable to jailbreak attacks.<n>We propose an improved safety alignment that disentangles DPO objectives into two components: (1) robust refusal training, which encourages refusal even when partial unsafe generations are produced, and (2) targeted unlearning of harmful knowledge.
  arXiv  Detail & Related papers  (2025-03-05T18:01:05Z)
• A Generative Approach to LLM Harmfulness Mitigation with Red Flag Tokens [26.119521867045616]
  We propose augmenting the model's vocabulary with a special red flag token.<n>We train the model to insert this token whenever harmful content is generated or imminent.<n>This approach is complementary to existing safety technique.
  arXiv  Detail & Related papers  (2025-02-22T21:48:48Z)
• Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoders [29.356200147371275]
  Large language models (LLMs) excel at handling human queries, but they can occasionally generate flawed or unexpected responses.<n>We propose using a fixed vocabulary set for feature interpretations and designing a mutual information-based objective.<n>We propose two runtime steering strategies that adjust the learned feature activations based on their corresponding explanations.
  arXiv  Detail & Related papers  (2025-02-21T16:36:42Z)
• SEER: Self-Explainability Enhancement of Large Language Models' Representations [18.840860385644316]
  We propose a self-explaining method SEER to explain Large Language Models (LLMs)<n>In this paper, we propose a self-explaining method SEER, enhancing LLMs' explainability by aggregating the same concept and disentangling the different concepts in the representation space.<n>We showcase the applications of SEER on trustworthiness-related tasks, where self-explained LLMs achieve consistent improvement in explainability and performance.
  arXiv  Detail & Related papers  (2025-02-07T13:25:33Z)
• Analyzing Finetuning Representation Shift for Multimodal LLMs Steering [56.710375516257876]
  We propose to map hidden states to interpretable visual and textual concepts.<n>This enables us to more efficiently compare certain semantic dynamics, such as the shift from an original and fine-tuned model.<n>We also demonstrate the use of shift vectors to capture these concepts changes.
  arXiv  Detail & Related papers  (2025-01-06T13:37:13Z)
• Aligning Large Language Models for Faithful Integrity Against Opposing Argument [71.33552795870544]
  Large Language Models (LLMs) have demonstrated impressive capabilities in complex reasoning tasks.<n>They can be easily misled by unfaithful arguments during conversations, even when their original statements are correct.<n>We propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation.
  arXiv  Detail & Related papers  (2025-01-02T16:38:21Z)
• Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal [21.342265570934995]
  Existing methods have largely overlooked the importance of refusal responses as a means of enhancing MLLMs reliability.<n>We present the Information Boundary-aware Learning Framework (InBoL), a novel approach that empowers MLLMs to refuse to answer user queries when encountering insufficient information.<n>This framework introduces a comprehensive data generation pipeline and tailored training strategies to improve the model's ability to deliver appropriate refusal responses.
  arXiv  Detail & Related papers  (2024-12-15T14:17:14Z)
• Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models [31.558429029429863]
  Large Language Models (LLMs) are expected to be predictable and trustworthy to support reliable decision-making systems.<n>This work examines logical preference consistency as a foundational requirement for building more dependable LLM systems.<n>We show that improving consistency leads to better performance in LLM-driven logic-based algorithms.
  arXiv  Detail & Related papers  (2024-10-03T04:34:04Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs)<n>We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention [53.896974148579346]
  Large Language Models (LLMs) have achieved unprecedented breakthroughs in various natural language processing domains. The enigmatic black-box'' nature of LLMs remains a significant challenge for interpretability, hampering transparent and accountable applications. We propose a novel methodology anchored in sparsity-guided techniques, aiming to provide a holistic interpretation of LLMs.
  arXiv  Detail & Related papers  (2023-12-22T19:55:58Z)
• Semantic Consistency for Assuring Reliability of Large Language Models [9.040736633675136]
  Large Language Models (LLMs) exhibit remarkable fluency and competence across various natural language tasks.<n>We introduce a general measure of semantic consistency, and formulate multiple versions of this metric to evaluate the performance of various LLMs.<n>We propose a novel prompting strategy, called Ask-to-Choose (A2C), to enhance semantic consistency.
  arXiv  Detail & Related papers  (2023-08-17T18:11:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.