Exploring Concreteness Through a Figurative Lens

• URL: http://arxiv.org/abs/2604.18296v1
• Date: Mon, 20 Apr 2026 14:02:42 GMT
• Title: Exploring Concreteness Through a Figurative Lens
• Abstract summary: We conduct a layer-wise and geometric analysis of hidden representations across four model families.<n>We find that LLMs separate literal and figurative usage in early layers, and that mid-to-late layers compress concreteness into a one-dimensional direction.<n>A single concreteness direction supports efficient figurative-language classification and enables training-free steering of generation toward more literal or more figurative rewrites.
• Score: 5.331858521587434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Static concreteness ratings are widely used in NLP, yet a word's concreteness can shift with context, especially in figurative language such as metaphor, where common concrete nouns can take abstract interpretations. While such shifts are evident from context, it remains unclear how LLMs understand concreteness internally. We conduct a layer-wise and geometric analysis of LLM hidden representations across four model families, examining how models distinguish literal vs figurative uses of the same noun and how concreteness is organized in representation space. We find that LLMs separate literal and figurative usage in early layers, and that mid-to-late layers compress concreteness into a one-dimensional direction that is consistent across models. Finally, we show that this geometric structure is practically useful: a single concreteness direction supports efficient figurative-language classification and enables training-free steering of generation toward more literal or more figurative rewrites.

Related papers

• CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations [0.0]
  CAMAUCMAR organizes meaning into nested lexical, cultural, and metaphorical embedding subspaces.<n>The geometric readout detects metaphor well above chance when the inter-layer geometry is shaped by paired supervision.<n>A controlled ablation shows that grounding the lexical layer in morphological roots gives a small but consistent gain.
  arXiv  Detail & Related papers  (2026-07-17T11:05:51Z)
• Floating or Suggesting Ideas? A Large-Scale Contrastive Analysis of Metaphorical and Literal Verb-Object Constructions [53.690096725532726]
  We analyze 297 English verb-object pairs (e.g., float idea vs. suggest idea) in 2M corpus sentences.<n>Using five NLP tools, we extract 2,293 cognitive and linguistic features capturing affective, lexical, syntactic, and discourse-level properties.<n>Cross-pair results show literal contexts have higher lexical frequency, cohesion, and structural regularity, while metaphorical contexts show greater affective load, imageability, lexical diversity, and constructional specificity.
  arXiv  Detail & Related papers  (2026-04-09T14:08:57Z)
• I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes [53.717874073948984]
  We evaluate eight state-of-the-art generative large language models (MLLMs) on their ability to detect and explain six types of figurative meaning.<n>Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present.
  arXiv  Detail & Related papers  (2026-03-24T14:02:48Z)
• Unveiling LLMs' Metaphorical Understanding: Exploring Conceptual Irrelevance, Context Leveraging and Syntactic Influence [40.32545329527664]
  Large Language Models (LLMs) demonstrate advanced capabilities in knowledge integration, contextual reasoning, and creative generation.<n>This study examines LLMs' metaphor-processing abilities from three perspectives.
  arXiv  Detail & Related papers  (2025-10-05T09:45:51Z)
• The Representational Alignment between Humans and Language Models is implicitly driven by a Concreteness Effect [4.491391835956324]
  We estimate semantic distances implicitly used by humans, for a set of carefully selected abstract and concrete nouns.<n>We find that the implicit representational space of participants and the semantic representations of language models are significantly aligned.<n>Results indicate that humans and language models converge on the concreteness dimension, but not on other dimensions.
  arXiv  Detail & Related papers  (2025-05-21T15:57:58Z)
• QUDsim: Quantifying Discourse Similarities in LLM-Generated Text [70.22275200293964]
  We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression.<n>We then use this framework to build $textbfQUDsim$, a similarity metric that can detect discursive parallels between documents.<n>Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs.
  arXiv  Detail & Related papers  (2025-04-12T23:46:09Z)
• Compositional Entailment Learning for Hyperbolic Vision-Language Models [54.41927525264365]
  We show how to fully leverage the innate hierarchical nature of hyperbolic embeddings by looking beyond individual image-text pairs.<n>We propose Compositional Entailment Learning for hyperbolic vision-language models.<n> Empirical evaluation on a hyperbolic vision-language model trained with millions of image-text pairs shows that the proposed compositional learning approach outperforms conventional Euclidean CLIP learning.
  arXiv  Detail & Related papers  (2024-10-09T14:12:50Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• LMs stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Models [0.0]
  Figurative language is a challenge for language models since its interpretation deviates from their conventional order and meaning. Yet, humans can easily understand and interpret metaphors as they can be derived from embodied metaphors. This study shows how larger language models perform better at interpreting metaphoric sentences when the action of the metaphorical sentence is more embodied.
  arXiv  Detail & Related papers  (2023-05-05T11:44:12Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)
• Grounded and Controllable Image Completion by Incorporating Lexical Semantics [111.47374576372813]
  Lexical Semantic Image Completion (LSIC) may have potential applications in art, design, and heritage conservation. We advocate generating results faithful to both visual and lexical semantic context. One major challenge for LSIC comes from modeling and aligning the structure of visual-semantic context.
  arXiv  Detail & Related papers  (2020-02-29T16:54:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.