Behavioral Analysis of Information Salience in Large Language Models

• URL: http://arxiv.org/abs/2502.14613v2
• Date: Tue, 27 May 2025 13:13:38 GMT
• Title: Behavioral Analysis of Information Salience in Large Language Models
• Authors: Jan Trienes, Jörg Schlötterer, Junyi Jessy Li, Christin Seifert,
• Abstract summary: We introduce an explainable framework to derive and investigate information salience in Large Language Models.<n>Experiments on 13 models across four datasets reveal that LLMs have a nuanced, hierarchical notion of salience, generally consistent across model families and sizes.<n>While models show highly consistent behavior and hence salience patterns, this notion of salience cannot be accessed through introspection, and only weakly correlates with human perceptions of information salience.
• Score: 36.80435135374382
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) excel at text summarization, a task that requires models to select content based on its importance. However, the exact notion of salience that LLMs have internalized remains unclear. To bridge this gap, we introduce an explainable framework to systematically derive and investigate information salience in LLMs through their summarization behavior. Using length-controlled summarization as a behavioral probe into the content selection process, and tracing the answerability of Questions Under Discussion throughout, we derive a proxy for how models prioritize information. Our experiments on 13 models across four datasets reveal that LLMs have a nuanced, hierarchical notion of salience, generally consistent across model families and sizes. While models show highly consistent behavior and hence salience patterns, this notion of salience cannot be accessed through introspection, and only weakly correlates with human perceptions of information salience.

Related papers

• What Matters to an LLM? Behavioral and Computational Evidences from Summarization [9.582572639590508]
  Large Language Models (LLMs) are now state-of-the-art at summarization, yet the internal notion of importance that drives their information selections remains hidden.<n>We propose to investigate this by combining behavioral and computational analyses.
  arXiv  Detail & Related papers  (2026-01-31T02:23:30Z)
• Do Reasoning Models Ask Better Questions? A Formal Information-Theoretic Analysis on Multi-Turn LLM Games [0.0]
  Large Language Models (LLMs) excel at many tasks but struggle with a critical ability for resolving ambiguity in user requests.<n>We propose a multi-turn dialogue framework that quantitatively measures how effectively LLMs gather information through yes/no questions.<n>Our experiments demonstrate that, among the evaluated models, the ones with explicit reasoning capabilities achieve higher IG per turn and reach solutions in fewer steps.
  arXiv  Detail & Related papers  (2026-01-25T06:38:15Z)
• Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit [16.056849135589324]
  Analyzing large-scale text corpora is a core challenge in machine learning.<n>We propose using sparse autoencoders (SAEs) to create SAE embeddings.<n>We show that SAE embeddings are more cost-effective and reliable than LLMs and more controllable than dense embeddings.
  arXiv  Detail & Related papers  (2025-12-10T21:26:24Z)
• Large Language Model Sourcing: A Survey [84.63438376832471]
  Large language models (LLMs) have revolutionized artificial intelligence, shifting from supporting objective tasks to empowering subjective decision-making.<n>Due to the black-box nature of LLMs and the human-like quality of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement become significant.<n>This survey presents a systematic investigation into provenance tracking for content generated by LLMs, organized around four interrelated dimensions.
  arXiv  Detail & Related papers  (2025-10-11T10:52:30Z)
• KScope: A Framework for Characterizing the Knowledge Status of Language Models [19.891459472894528]
  We introduce a taxonomy of five knowledge statuses based on the consistency and correctness of LLM knowledge modes.<n>We then propose KScope, a hierarchical framework of statistical tests that progressively refines hypotheses about knowledge modes.
  arXiv  Detail & Related papers  (2025-06-09T06:06:05Z)
• Can Large Language Models Trigger a Paradigm Shift in Travel Behavior Modeling? Experiences with Modeling Travel Satisfaction [2.2974830861901414]
  This study uses data on travel satisfaction from a household survey in shanghai to identify the existence and source of misalignment between Large Language Models and human behavior.<n>We find that the zero-shot LLM exhibits behavioral misalignment, resulting in relatively low prediction accuracy.<n>We propose an LLM-based modeling approach that can be applied to model travel behavior using samples of small sizes.
  arXiv  Detail & Related papers  (2025-05-29T09:11:58Z)
• Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis [8.725781605542675]
  Large Language Models (LLMs) achieve remarkable performance through training on massive datasets.<n>LLMs can exhibit concerning behaviors such as verbatim reproduction of training data rather than true generalization.<n>This paper introduces PEARL, a novel approach for detecting memorization in LLMs.
  arXiv  Detail & Related papers  (2025-05-05T20:42:34Z)
• Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations [12.615208274851152]
  Large Language Models (LLMs) are often asked to explain their outputs to enhance accuracy and transparency. Evidence suggests that these explanations can misrepresent the models' true reasoning processes. This paper introduces, cross-examiner, a new method for generating follow-up questions based on a model's explanation of an initial question.
  arXiv  Detail & Related papers  (2025-03-11T18:50:43Z)
• Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models [30.066436019078164]
  We study what kind of generalisation strategies Large Language Models employ when performing reasoning tasks. Our findings indicate that the approach to reasoning the models use is unlike retrieval, and more like a generalisable strategy.
  arXiv  Detail & Related papers  (2024-11-19T15:47:12Z)
• Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors [74.04775677110179]
  In-context Learning (ICL) has become the primary method for performing natural language tasks with Large Language Models (LLMs)<n>In this work, we examine whether this is the result of the aggregation used in corresponding datasets, where trying to combine low-agreement, disparate annotations might lead to annotation artifacts that create detrimental noise in the prompt.<n>Our results indicate that aggregation is a confounding factor in the modeling of subjective tasks, and advocate focusing on modeling individuals instead.
  arXiv  Detail & Related papers  (2024-10-17T17:16:00Z)
• Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language [0.0]
  We use a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios.
  arXiv  Detail & Related papers  (2024-10-03T13:25:15Z)
• Estimating Knowledge in Large Language Models Without Generating a Single Token [12.913172023910203]
  Current methods to evaluate knowledge in large language models (LLMs) query the model and then evaluate its generated responses. In this work, we ask whether evaluation can be done before the model has generated any text. Experiments with a variety of LLMs show that KEEN, a simple probe trained over internal subject representations, succeeds at both tasks.
  arXiv  Detail & Related papers  (2024-06-18T14:45:50Z)
• Optimizing Language Model's Reasoning Abilities with Weak Supervision [48.60598455782159]
  We present textscPuzzleBen, a weakly supervised benchmark that comprises 25,147 complex questions, answers, and human-generated rationales. A unique aspect of our dataset is the inclusion of 10,000 unannotated questions, enabling us to explore utilizing fewer supersized data to boost LLMs' inference capabilities.
  arXiv  Detail & Related papers  (2024-05-07T07:39:15Z)
• Eliciting Personality Traits in Large Language Models [0.0]
  Large Language Models (LLMs) are increasingly being utilized by both candidates and employers in the recruitment context. This study seeks to obtain a better understanding of such models by examining their output variations based on different input prompts.
  arXiv  Detail & Related papers  (2024-02-13T10:09:00Z)
• Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization [66.4659448305396]
  This study analyzes various LMs with three probing-based experiments to shed light on the reasons behind the In- vs. Cross-Topic generalization gap. We demonstrate, for the first time, that generalization gaps and the robustness of the embedding space vary significantly across LMs.
  arXiv  Detail & Related papers  (2024-02-02T12:59:27Z)
• On Context Utilization in Summarization with Large Language Models [83.84459732796302]
  Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. We conduct the first comprehensive study on context utilization and position bias in summarization.
  arXiv  Detail & Related papers  (2023-10-16T16:45:12Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.