Latent Structure of Affective Representations in Large Language Models

• URL: http://arxiv.org/abs/2604.07382v2
• Date: Sat, 11 Apr 2026 22:20:55 GMT
• Title: Latent Structure of Affective Representations in Large Language Models
• Abstract summary: We investigate the latent structure of affective representations in large language models (LLMs) using geometric data analysis tools.<n>First, we show that LLMs learn coherent latent representations of affective emotions that align with widely used valence--arousal models from psychology.<n>Second, we find that these representations exhibit nonlinear geometric structure that can nonetheless be well-approximated linearly.<n>Third, we demonstrate that the learned latent representation space can be leveraged to quantify uncertainty in emotion processing tasks.
• Score: 6.345340156849189
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The geometric structure of latent representations in large language models (LLMs) is an active area of research, driven in part by its implications for model transparency and AI safety. Existing literature has focused mainly on general geometric and topological properties of the learnt representations, but due to a lack of ground-truth latent geometry, validating the findings of such approaches is challenging. Emotion processing provides an intriguing testbed for probing representational geometry, as emotions exhibit both categorical organization and continuous affective dimensions, which are well-established in the psychology literature. Moreover, understanding such representations carries safety relevance. In this work, we investigate the latent structure of affective representations in LLMs using geometric data analysis tools. We present three main findings. First, we show that LLMs learn coherent latent representations of affective emotions that align with widely used valence--arousal models from psychology. Second, we find that these representations exhibit nonlinear geometric structure that can nonetheless be well-approximated linearly, providing empirical support for the linear representation hypothesis commonly assumed in model transparency methods. Third, we demonstrate that the learned latent representation space can be leveraged to quantify uncertainty in emotion processing tasks. Our findings suggest that LLMs acquire affective representations with geometric structure paralleling established models of human emotion, with practical implications for model interpretability and safety.

Related papers

• Abstract representational geometry supports inference in large language models [9.40587716832914]
  Internal mechanisms of large language models remain largely opaque, making it unclear whether they form comparable abstract representations or rely on task-specific statistical regularities when performing comparable reasoning tasks.<n>We report that although LLMs exhibit generalizable reasoning less frequently than humans, when such inference occurs, their internal states exhibit abstract geometric structures that resemble those reported in the hippocampus.
  arXiv  Detail & Related papers  (2026-06-22T13:50:34Z)
• Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language [60.70116693224378]
  We design a unified formal language that integrates plane and solid geometry, comprehensively covering geometric structures and semantic relations.<n>We construct GDP-29K, a large-scale dataset comprising 20k plane and 9k solid geometry samples collected from diverse real-world sources.
  arXiv  Detail & Related papers  (2026-04-13T15:09:56Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Are Emotions Arranged in a Circle? Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learning [18.807120508172407]
  This paper proposes a method to induce circular emotion representations within language model embeddings via contrastive learning on a hypersphere.<n>We show that while this circular alignment offers superior interpretability and robustness against dimensionality reduction, it underperforms compared to conventional designs in high-dimensional settings and fine-grained classification.
  arXiv  Detail & Related papers  (2026-01-10T13:54:06Z)
• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling [68.14113731953971]
  This paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like imagination.<n>We show that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks.
  arXiv  Detail & Related papers  (2025-12-01T16:01:41Z)
• Less is More: Local Intrinsic Dimensions of Contextual Language Models [15.934492316122025]
  We introduce a novel perspective based on the geometric properties of contextual latent embeddings to study the effects of training and fine-tuning.<n>We show that the local dimensions provide insights into the model's training dynamics and generalization ability.<n>Our experiments suggest configuring a practical: reductions in the mean local dimension tend to accompany and predict subsequent performance gains.
  arXiv  Detail & Related papers  (2025-06-01T14:30:46Z)
• The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology [4.280045926995889]
  This study focuses on how adversarial inputs systematically affect the internal representation spaces of Large Language Models.<n>By quantifying the shape of activations and neuronal information flow, our architecture-agnostic framework reveals fundamental invariants of representational change.
  arXiv  Detail & Related papers  (2025-05-26T18:31:49Z)
• MATHGLANCE: Multimodal Large Language Models Do Not Know Where to Look in Mathematical Diagrams [65.02628814094639]
  Diagrams serve as a fundamental form of visual language, representing complex concepts and their inter-relationships through structured symbols, shapes, and spatial arrangements.<n>Current benchmarks conflate perceptual and reasoning tasks, making it difficult to assess whether Multimodal Large Language Models genuinely understand mathematical diagrams beyond superficial pattern recognition.<n>We introduce MATHGLANCE, a benchmark specifically designed to isolate and evaluate mathematical perception in MLLMs.<n>We construct GeoPeP, a perception-oriented dataset of 200K structured geometry image-text annotated with geometric primitives and precise spatial relationships.
  arXiv  Detail & Related papers  (2025-03-26T17:30:41Z)
• Geometry Distributions [51.4061133324376]
  We propose a novel geometric data representation that models geometry as distributions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity.
  arXiv  Detail & Related papers  (2024-11-25T04:06:48Z)
• Learning Visual-Semantic Subspace Representations [49.17165360280794]
  We introduce a nuclear norm-based loss function, grounded in the same information theoretic principles that have proved effective in self-supervised learning.<n>We present a theoretical characterization of this loss, demonstrating that, in addition to promoting classity, it encodes the spectral geometry of the data within a subspace lattice.
  arXiv  Detail & Related papers  (2024-05-25T12:51:38Z)
• Using Shapley interactions to understand how models use structure [9.93791871599425]
  We use Shapley Taylor interaction indices (STII) to examine how language and speech models internally relate and structure their inputs.<n>We relate the interaction patterns in models to three underlying linguistic structures: syntactic structure, non-compositional semantics, and phonetic coarticulation.
  arXiv  Detail & Related papers  (2024-03-19T19:13:22Z)
• Analyzing the Representational Geometry of Acoustic Word Embeddings [22.677210029168588]
  Acoustic word embeddings (AWEs) are vector representations such that different acoustic exemplars of the same word are projected nearby. This paper takes a closer analytical look at AWEs learned from English speech and study how the choice of the learning objective and the architecture shapes their representational profile.
  arXiv  Detail & Related papers  (2023-01-08T10:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.