Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes

• URL: http://arxiv.org/abs/2608.07208v1
• Date: Fri, 07 Aug 2026 13:21:50 GMT
• Title: Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes
• Abstract summary: Existing measures of how much a text is about a concept read the surface of the text.<n>Recent work has shown that a gap exists in what Large Language Models (LLMs) know internally versus what they express in their response.<n>This paper asks whether that internal knowledge, read by monitoring the activations of frozen, out-of-the-box LLMs, can stand in for task-specific fine-tuning.
• Score: 18.346517033220284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities. They score the words a text uses, not the judgment a reader forms about it. Recent work has shown that a gap exists in what Large Language Models (LLMs) know internally versus what they express in their response. This paper asks whether that internal knowledge, read by monitoring the activations of frozen, out-of-the-box LLMs, can stand in for task-specific fine-tuning when measuring concept content, and which extraction method reads it best. We extract such measures via the Recursive Feature Machine (RFM) algorithm and via linear probing, and compare these against an embedding baseline, surface baselines, and the same model's own answer to the question. We demonstrate the approach on financial text, a domain studied extensively and served by established annotated resources, using a human-annotated Environmental, Social and Governance (ESG) dataset. The best linear probe comes within 0.6 percentage points of a fine-tuned domain classifier's accuracy without any task-specific fine-tuning, and outscores the same model's own answer to the question in eleven of twelve comparisons, so the activations carry concept content the response does not report. The simple probe consistently beats the RFM concept vectors, which in turn provide what classification alone does not: a continuous score intended to reflect how strongly a concept is present in a text, whose validation awaits graded labels.

Related papers

• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• Context is Enough: Empirical Validation of $\textit{Sequentiality}$ on Essays [1.338174941551702]
  We show that the contextual version of sequentiality aligns more closely with human assessments of discourse-level traits.<n>Our findings support the use of context-based sequentiality as a validated, interpretable, and complementary feature for automated essay scoring and related NLP tasks.
  arXiv  Detail & Related papers  (2025-11-12T10:31:07Z)
• Regularization Through Reasoning: Systematic Improvements in Language Model Classification via Explanation-Enhanced Fine-Tuning [2.247737938202007]
  We evaluate whether attaching brief explanations to each label during fine-tuning yields better models.<n>We replace human-written explanations with text that is syntactically incoherent yet vocabulary-aligned with the originals.<n>The effect persists across datasets and training seeds, indicating that gains arise less from meaning than from structure.
  arXiv  Detail & Related papers  (2025-11-03T20:25:42Z)
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns [50.401907401444404]
  Large language models (LLMs) are crucial for preventing misuse and building trustworthy AI systems.<n>We propose RepreGuard, an efficient statistics-based detection method.<n> Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios.
  arXiv  Detail & Related papers  (2025-08-18T17:59:15Z)
• GUM-SAGE: A Novel Dataset and Approach for Graded Entity Salience Prediction [12.172254885579706]
  Graded entity salience assigns entities scores that reflect their relative importance in a text.<n>We introduce a novel approach for graded entity salience that combines the strengths of both approaches.<n>Our approach shows stronger correlation with scores based on human summaries and alignments, and outperforms existing techniques.
  arXiv  Detail & Related papers  (2025-04-15T01:26:14Z)
• On Context Utilization in Summarization with Large Language Models [83.84459732796302]
  Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. We conduct the first comprehensive study on context utilization and position bias in summarization.
  arXiv  Detail & Related papers  (2023-10-16T16:45:12Z)
• Textual Entailment Recognition with Semantic Features from Empirical Text Representation [60.31047947815282]
  A text entails a hypothesis if and only if the true value of the hypothesis follows the text. In this paper, we propose a novel approach to identifying the textual entailment relationship between text and hypothesis. We employ an element-wise Manhattan distance vector-based feature that can identify the semantic entailment relationship between the text-hypothesis pair.
  arXiv  Detail & Related papers  (2022-10-18T10:03:51Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application [63.10266319378212]
  We propose a method for measuring complex variables on a continuous, interval spectrum by combining supervised deep learning with the Constructing Measures approach to faceted Rasch item response theory (IRT) We demonstrate this new method on a dataset of 50,000 social media comments sourced from YouTube, Twitter, and Reddit and labeled by 11,000 U.S.-based Amazon Mechanical Turk workers.
  arXiv  Detail & Related papers  (2020-09-22T02:15:05Z)
• Word Embedding-based Text Processing for Comprehensive Summarization and Distinct Information Extraction [1.552282932199974]
  We propose two automated text processing frameworks specifically designed to analyze online reviews. The first framework is to summarize the reviews dataset by extracting essential sentence. The second framework is based on a question-answering neural network model trained to extract answers to multiple different questions.
  arXiv  Detail & Related papers  (2020-04-21T02:43:31Z)
• Extending Text Informativeness Measures to Passage Interestingness Evaluation (Language Model vs. Word Embedding) [1.2998637003026272]
  This paper defines the concept of Interestingness as a generalization of Informativeness. We then study the ability of state of the art Informativeness measures to cope with this generalization. We prove that the CLEF-INEX Tweet Contextualization 2012 Logarithm Similarity measure provides best results.
  arXiv  Detail & Related papers  (2020-04-14T18:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.