The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models

• URL: http://arxiv.org/abs/2601.13358v1
• Date: Mon, 19 Jan 2026 19:53:37 GMT
• Title: The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models
• Abstract summary: We analyze 25,000+ chain-of-thought trajectories across four domains (Law, Science, Code, Math)<n>We discover that neural scaling laws trigger domain-specific phase transitions rather than uniform capability gains.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scale does not uniformly improve reasoning - it restructures it. Analyzing 25,000+ chain-of-thought trajectories across four domains (Law, Science, Code, Math) and two scales (8B, 70B parameters), we discover that neural scaling laws trigger domain-specific phase transitions rather than uniform capability gains. Legal reasoning undergoes Crystallization: 45% collapse in representational dimensionality (d95: 501 -> 274), 31% increase in trajectory alignment, and 10x manifold untangling. Scientific and mathematical reasoning remain Liquid - geometrically invariant despite 9x parameter increase. Code reasoning forms a discrete Lattice of strategic modes (silhouette: 0.13 -> 0.42). This geometry predicts learnability. We introduce Neural Reasoning Operators - learned mappings from initial to terminal hidden states. In crystalline legal reasoning, our operator achieves 63.6% accuracy on held-out tasks via probe decoding, predicting reasoning endpoints without traversing intermediate states. We further identify a universal oscillatory signature (coherence ~ -0.4) invariant across domains and scales, suggesting attention and feedforward layers drive reasoning through opposing dynamics. These findings establish that the cost of thought is determined not by task difficulty but by manifold geometry - offering a blueprint for inference acceleration where topology permits.

Related papers

• Transformed in Translation: Two-Stage Structural Uncertainty in LLM-Based Scientific Autoformalization [0.0]
  Autoformalization turns accounts into executable mathematics, but executable code does not settle which model has been constructed.<n>We examine two sources of structural uncertainty: the formalizer that generates a response law, and recurrence that turns that law into trajectories.
  arXiv  Detail & Related papers  (2026-09-13T21:47:46Z)
• MindTopo: Can Foundation Models Reason in Topological Space? [52.56750995613252]
  We introduce MindTopo, a benchmark of topological intuition across five properties grounded in cognitive science and formal topology.<n>MindTopo evaluates each property at two cognitive levels. Reasoning asks a model to identify topological relations or infer how they change.<n>Every MLLM performs better on reasoning than on planning, and the best-performing model remains far below observed human performance.
  arXiv  Detail & Related papers  (2026-09-10T17:54:32Z)
• Trajectory Geometry of Transformer Representations Across Layers [0.5599792629509229]
  We recast the transformer forward pass as a discrete population trajectory through a high-dimensional representation manifold.<n>We characterize trajectory geometry using five metrics computed directly in the ambient space.
  arXiv  Detail & Related papers  (2026-06-08T09:54:07Z)
• Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Models [0.0]
  We show that representational geometry is consistently log-compressive across three magnitude domains.<n>One model produces a human-range Weber fraction (WF = 0.20) while the other does not, and both models perform at chance on temporal and spatial discrimination.<n> causal intervention reveals a layer dissociation: early layers are functionally implicated in magnitude processing (4.1x specificity) while later layers where geometry is strongest are not causally engaged.
  arXiv  Detail & Related papers  (2026-03-21T04:38:30Z)
• Anatomy of Capability Emergence: Scale-Invariant Representation Collapse and Top-Down Reorganization in Neural Networks [1.5567685129899713]
  We track five geometric measures across five model scales.<n>We find that training begins with a universal representation collapse to task-specific floors.<n>On Pythia, global geometric patterns replicate but per-task precursor signals do not.
  arXiv  Detail & Related papers  (2026-02-17T20:39:02Z)
• Milestones over Outcome: Unlocking Geometric Reasoning with Sub-Goal Verifiable Reward [67.00373428443879]
  We introduce a paradigm shift towards subgoal-level evaluation and learning.<n>We first construct GeoGoal, a benchmark synthesized via a rigorous formal verification data engine.<n>We propose the Sub-Goal Verifiable Reward (SGVR) framework, which replaces sparse signals with dense rewards based on the Skeleton Rate.
  arXiv  Detail & Related papers  (2026-01-08T16:17:56Z)
• Beyond Accuracy: A Geometric Stability Analysis of Large Language Models in Chess Evaluation [8.694064926527902]
  This paper argues that standard accuracy metrics fail to distinguish between genuine geometric reasoning and the superficial memorization of canonical board states.<n>We apply a novel evaluation methodology that rigorously tests model consistency under invariant transformations.<n>We conclude that geometric stability provides an essential metric for AI evaluation, offering a necessary proxy for disentangling reasoning capabilities from data contamination.
  arXiv  Detail & Related papers  (2025-12-17T02:49:10Z)
• Pay Attention Later: From Vector Space Diffusion to Linearithmic Spectral Phase-Locking [0.0]
  Standard Transformers suffer from a "Semantic Alignment Tax"<n>We introduce the Phase-Resonant Intelligent Spectral Model (PRISM)<n>PRISM encodes semantic identity as resonant frequencies in the complex domain (Cd) and replaces quadratic self-attention with linearithmic O(N log N) Gated Harmonic Convolutions.
  arXiv  Detail & Related papers  (2025-12-01T02:46:15Z)
• Geometrically-Constrained Agent for Spatial Reasoning [53.93718394870856]
  Vision Language Models exhibit a fundamental semantic-to-geometric gap in spatial reasoning.<n>Current paradigms fail to bridge this gap.<n>We propose a training-free agentic paradigm that resolves this gap by introducing a formal task constraint.
  arXiv  Detail & Related papers  (2025-11-27T17:50:37Z)
• Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer [0.8738725605667471]
  Chain-of-thought (CoT) reasoning has enabled transformer-based language models to excel at complex mathematics and multi-step planning.<n>In standard decoder-only architectures, these reasoning steps are externalized in natural language, improving interpretability at the cost of efficiency.<n>We investigate whether such reasoning structures emerge in Huginn-3.5B, a depth-recurrent Transformer that reuses layers at inference time without increasing parameter count.
  arXiv  Detail & Related papers  (2025-07-02T23:35:21Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• Do Larger Language Models Generalize Better? A Scaling Law for Implicit Reasoning at Pretraining Time [73.22651918134808]
  This work shows counterintuitive effects of model size scaling and provides new insights into the relationship between scaling and reasoning in language models (LMs)<n>We pretrain LMs from scratch on a synthetic implicit multihop reasoning environment designed to replicate the structure and distribution of real-world large-scale knowledge graphs.<n>We then assess the LMs' ability to complete the missing edges in the graph, which requires multi-hop reasoning that can be viewed as a simplification of implicit reasoning during real-world pretraining.
  arXiv  Detail & Related papers  (2025-04-04T17:57:22Z)
• ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates [51.633266497799745]
  hierarchical LLM reasoning via scaling thought templates can effectively optimize the reasoning search space.<n>We introduce three innovations: (i) a structured and generic thought template library, containing around 500 high-level thought templates capable of generalizing to similar or relevant reasoning problems; (ii) performing hierarchical reinforcement learning on a sequence of thought templates instead of long CoTs; and (iii) a brand new inference scaling system.
  arXiv  Detail & Related papers  (2025-02-10T18:51:47Z)
• Bridging Geometric States via Geometric Diffusion Bridge [79.60212414973002]
  We introduce the Geometric Diffusion Bridge (GDB), a novel generative modeling framework that accurately bridges initial and target geometric states. GDB employs an equivariant diffusion bridge derived by a modified version of Doob's $h$-transform for connecting geometric states. We show that GDB surpasses existing state-of-the-art approaches, opening up a new pathway for accurately bridging geometric states.
  arXiv  Detail & Related papers  (2024-10-31T17:59:53Z)
• On the Convergence of Gradient Descent for Large Learning Rates [55.33626480243135]
  We show that convergence is impossible when a fixed step size is used.<n>We provide a proof of this in the case of linear neural networks with a squared loss.<n>We also prove the impossibility of convergence for more general losses without requiring strong assumptions such as Lipschitz continuity for the gradient.
  arXiv  Detail & Related papers  (2024-02-20T16:01:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.