On the Persistent Effects of Lexicality in Large Language Models

• URL: http://arxiv.org/abs/2606.02750v2
• Date: Wed, 03 Jun 2026 18:55:08 GMT
• Title: On the Persistent Effects of Lexicality in Large Language Models
• Abstract summary: We quantify the effect of lexical overlap relative to semantic content in large language models (LLMs)<n>We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions.<n>We observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning.
• Score: 19.762746965467635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited. In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.

Related papers

• Beyond Via: Analysis and Estimation of the Impact of Large Language Models in Academic Papers [10.297544444218685]
  We report several shifts in word usage that are likely driven by large language models (LLMs)<n>Experiments show that current classifiers struggle to accurately determine which specific model generated a given text in multi-class classification tasks.
  arXiv  Detail & Related papers  (2026-03-26T16:49:00Z)
• Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs [100.02824137397464]
  We investigate how Large Language Models adapt their internal representations when encountering inputs of increasing difficulty.<n>We reveal a consistent and quantifiable phenomenon: as task difficulty increases, the last hidden states of LLMs become substantially sparser.<n>This sparsity--difficulty relation is observable across diverse models and domains.
  arXiv  Detail & Related papers  (2026-03-03T18:48:15Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models [28.102903742881576]
  We introduce a framework for abstraction-guided reasoning that explicitly separates structural inference from lexical semantics.<n>We show that abstraction-aligned steering reduces content-driven errors and improves validity-sensitive performance.
  arXiv  Detail & Related papers  (2026-02-02T18:48:44Z)
• Explaining multimodal LLMs via intra-modal token interactions [55.27436637894534]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet their internal decision-making mechanisms remain insufficiently understood.<n>We propose enhancing interpretability by leveraging intra-modal interaction.
  arXiv  Detail & Related papers  (2025-09-26T14:39:13Z)
• CALE : Concept-Aligned Embeddings for Both Within-Lemma and Inter-Lemma Sense Differentiation [0.0]
  Lexical semantics is concerned with both the multiple senses a word can adopt in different contexts, and the semantic relations that exist between meanings of different words.<n>To investigate them, Contextualized Language Models are a valuable tool that provides context-sensitive representations.<n>We propose an extension, Concept Differentiation, to include inter-words scenarios.
  arXiv  Detail & Related papers  (2025-08-06T14:43:22Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps [7.342347950764399]
  We investigate the role of various demonstration components in the in-context learning performance of large language models (LLMs) Specifically, we explore the impacts of ground-truth labels, input distribution, and complementary explanations, particularly when these are altered or perturbed.
  arXiv  Detail & Related papers  (2023-07-11T07:03:29Z)
• Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis [13.07356367140208]
  We introduce DistNLI, a new diagnostic dataset for natural language inference. We find that the extent of models' understanding is associated with model size and vocabulary size.
  arXiv  Detail & Related papers  (2022-09-11T00:33:28Z)
• Image Synthesis via Semantic Composition [74.68191130898805]
  We present a novel approach to synthesize realistic images based on their semantic layouts. It hypothesizes that for objects with similar appearance, they share similar representation. Our method establishes dependencies between regions according to their appearance correlation, yielding both spatially variant and associated representations.
  arXiv  Detail & Related papers  (2021-09-15T02:26:07Z)
• Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge [59.22170796793179]
  Transformers Language Models (TLMs) were tested on a benchmark for the textitdynamic estimation of thematic fit Our results show that TLMs can reach performances that are comparable to those achieved by SDM. However, additional analysis consistently suggests that TLMs do not capture important aspects of event knowledge.
  arXiv  Detail & Related papers  (2021-07-22T20:52:26Z)
• Interpretable Time-series Representation Learning With Multi-Level Disentanglement [56.38489708031278]
  Disentangle Time Series (DTS) is a novel disentanglement enhancement framework for sequential data. DTS generates hierarchical semantic concepts as the interpretable and disentangled representation of time-series. DTS achieves superior performance in downstream applications, with high interpretability of semantic concepts.
  arXiv  Detail & Related papers  (2021-05-17T22:02:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.