Reasoning emerges from constrained inference manifolds in large language models

• URL: http://arxiv.org/abs/2605.08142v1
• Date: Sat, 02 May 2026 10:41:20 GMT
• Title: Reasoning emerges from constrained inference manifolds in large language models
• Abstract summary: We study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference.<n>We find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning.
• Score: 98.6966322378456
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment.

Related papers

• Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine [60.669081685261965]
  Diffusion models generate high-dimensional data with remarkable quality.<n>How their training efficiently learns the score function remains theoretically unexplained.<n>We instantiate this principle as Score-induced Latent Diffusion (SiLD)
  arXiv  Detail & Related papers  (2026-05-16T16:51:10Z)
• Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models [0.0]
  Internal representations carry correctness-related signals, but their coarse aggregations may obscure the token and layer structure underlying reasoning.<n>We investigate hidden-state transitions across decoding steps and layers, and identify a pattern in large reasoning models (LRMs)<n>We formalize this characteristic as Stemporal Amplitude of Latent Transition (StALT), a statistic trajectory that summarizes temporal changes between adjacent tokens weighted by within-token layer saliency.
  arXiv  Detail & Related papers  (2026-05-03T12:46:41Z)
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments [75.60795462502949]
  SpatialEvo is a self-evolving framework for 3D spatial reasoning.<n>It formalizes 16 spatial reasoning task categories under explicit geometric validation rules.<n>It converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback.
  arXiv  Detail & Related papers  (2026-04-15T17:59:12Z)
• Emergent Manifold Separability during Reasoning in Large Language Models [46.78826734548872]
  Chain-of-Thought prompting significantly improves reasoning in Large Language Models.<n>We quantify the linear separability of latent representations without the confounding factors of probe training.
  arXiv  Detail & Related papers  (2026-02-23T20:36:17Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models [0.8694591156258423]
  Large language models perform text generation through high-dimensional internal dynamics.<n>Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored.<n>We discuss a composite dynamical metric, computed from activation time-series during autoregressive generation.
  arXiv  Detail & Related papers  (2026-01-11T21:57:52Z)
• The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology [4.280045926995889]
  This study focuses on how adversarial inputs systematically affect the internal representation spaces of Large Language Models.<n>By quantifying the shape of activations and neuronal information flow, our architecture-agnostic framework reveals fundamental invariants of representational change.
  arXiv  Detail & Related papers  (2025-05-26T18:31:49Z)
• Sequential Representation Learning via Static-Dynamic Conditional Disentanglement [58.19137637859017]
  This paper explores self-supervised disentangled representation learning within sequential data, focusing on separating time-independent and time-varying factors in videos. We propose a new model that breaks the usual independence assumption between those factors by explicitly accounting for the causal relationship between the static/dynamic variables. Experiments show that the proposed approach outperforms previous complex state-of-the-art techniques in scenarios where the dynamics of a scene are influenced by its content.
  arXiv  Detail & Related papers  (2024-08-10T17:04:39Z)
• Calibrating Reasoning in Language Models with Internal Consistency [18.24350001344488]
  Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks.<n>LLMs often generate text with obvious mistakes and contradictions.<n>In this work, we investigate reasoning in LLMs through the lens of internal representations.
  arXiv  Detail & Related papers  (2024-05-29T02:44:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.