Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

• URL: http://arxiv.org/abs/2608.17950v2
• Date: Fri, 21 Aug 2026 00:47:13 GMT
• Title: Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
• Abstract summary: This work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.<n>We show that factually grounded generations maintain structural integrity with their source context, whereas hallucinations induce severe topological collapse.
• Score: 1.8190501646350212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, we trace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the "Six Degrees of Separation" limit (=< 6 semantic hops). Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (approximately 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.

Related papers

• DeSeG: Decoupling Semantic Intent and Geometric Constraints for Physically Plausible Human-Scene Interaction [50.06148080975928]
  We propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints.<n>DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
  arXiv  Detail & Related papers  (2026-07-07T03:21:08Z)
• What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis [24.026935801991744]
  We analyze Latent Chain-of-Thought (CoT) from an information-theoretic perspective and identify this failure as a dual collapse.<n>To measure these effects, we introduce the Unified Latent Probe (ULP), which quantifies the mutual information between latent trajectories and explicit reasoning steps.
  arXiv  Detail & Related papers  (2026-06-18T10:51:53Z)
• Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning [11.05919811646786]
  We propose RIS (Retrieve, Integrate, and Synthesize), a spatial-semantic grounded framework that develops latent reasoning as a compatible extension of pretrained MLLM computation.<n>RIS anchors latent tokens to both spatial and semantic evidence, enforces their causal role through a progressive attention bottleneck, and introduces short language transition tokens to bridge synthesized latent states back to vocabulary-aligned decoding.
  arXiv  Detail & Related papers  (2026-05-08T01:33:58Z)
• SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning [67.67774742200626]
  spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, is a core challenge for large language models.<n>We propose a unified multimodal generation framework that integrates textual reasoning with visual imagination.<n>Our framework adopts a divide-and-conquer strategy, using text chain-of-thought for high-level semantic planning and the visual imagination for geometry-sensitive state transformation and consistency preservation.
  arXiv  Detail & Related papers  (2026-04-19T11:21:59Z)
• LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval [74.72139580745511]
  LaSER is a novel self-distillation framework that internalizes explicit reasoning into the latent space of retrievers.<n>Our method successfully combines the reasoning depth of explicit CoT pipelines with the inference efficiency of standard dense retrievers.
  arXiv  Detail & Related papers  (2026-03-02T04:11:18Z)
• Multimodal Latent Reasoning via Hierarchical Visual Cues Injection [16.779425236020433]
  This work posits that robust reasoning should evolve within a latent space, integrating multimodal signals seamlessly.<n>We propose a novel framework that instills deliberate, "slow thinking" without depending on superficial textual rationales.<n>We show that test-time scaling is effective when incorporating vision knowledge, and that integrating hierarchical information significantly enhances the model's understanding of complex scenes.
  arXiv  Detail & Related papers  (2026-02-05T06:31:12Z)
• Higher-Order Knowledge Representations for Agentic Scientific Reasoning [1.1458853556386797]
  We introduce a methodology for constructing hypergraph-based knowledge representations that faithfully encode multi-entity relationships.<n> Applied to a corpus of 1,100 manuscripts on biocomposite scaffolds, our framework constructs a global hypergraph of 161,172 nodes and 320,201 hyperedges.<n>We further demonstrate that equipping agentic systems with hypergraph tools, specifically using node-intersection constraints, enables them to bridge semantically distant concepts.
  arXiv  Detail & Related papers  (2026-01-08T12:25:37Z)
• GeoGNN: Quantifying and Mitigating Semantic Drift in Text-Attributed Graphs [59.61242815508687]
  Graph neural networks (GNNs) on text--attributed graphs (TAGs) encode node texts using pretrained language models (PLMs) and propagate these embeddings through linear neighborhood aggregation.<n>This work introduces a local PCA-based metric that measures the degree of semantic drift and provides the first quantitative framework to analyze how different aggregation mechanisms affect manifold structure.
  arXiv  Detail & Related papers  (2025-11-12T06:48:43Z)
• A Survey on Latent Reasoning [100.54120559169735]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities.<n>CoT reasoning that verbalizes intermediate steps limits the model's expressive bandwidth.<n>Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state.
  arXiv  Detail & Related papers  (2025-07-08T17:29:07Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.