How to Tame Your LLM: Semantic Collapse in Continuous Systems

• URL: http://arxiv.org/abs/2512.05162v1
• Date: Thu, 04 Dec 2025 11:33:02 GMT
• Title: How to Tame Your LLM: Semantic Collapse in Continuous Systems
• Authors: C. M. Wyss,
• Abstract summary: We develop a theory of semantic dynamics for large language models by formalizing them as Continuous State Machines (CSMs)<n>We prove the Semantic characterization Theorem (SCT)<n>We extend the SCT to drifting kernels and adiabatic settings, showing that slowly preserving compactness, spectral coherence, and basin structure.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop a general theory of semantic dynamics for large language models by formalizing them as Continuous State Machines (CSMs): smooth dynamical systems whose latent manifolds evolve under probabilistic transition operators. The associated transfer operator $P: L^2(M,μ) \to L^2(M,μ)$ encodes the propagation of semantic mass. Under mild regularity assumptions (compactness, ergodicity, bounded Jacobian), $P$ is compact with discrete spectrum. Within this setting, we prove the Semantic Characterization Theorem (SCT): the leading eigenfunctions of $P$ induce finitely many spectral basins of invariant meaning, each definable in an o-minimal structure over $\mathbb{R}$. Thus spectral lumpability and logical tameness coincide. This explains how discrete symbolic semantics can emerge from continuous computation: the continuous activation manifold collapses into a finite, logically interpretable ontology. We further extend the SCT to stochastic and adiabatic (time-inhomogeneous) settings, showing that slowly drifting kernels preserve compactness, spectral coherence, and basin structure.

Related papers

• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds [0.5729426778193398]
  We study the emergence of multi-step reasoning in deep Transformer language models through a geometric and statistical-physics lens.<n>We formalize the forward pass as a discrete coarse-graining map and relate the appearance of stable "concept basins" to fixed points of this renormalization-like dynamics.<n>The resulting low-entropy regime is characterized by a spectral tail collapse and by the formation of transient, reusable object-like structures in representation space.
  arXiv  Detail & Related papers  (2026-01-16T23:11:02Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction [54.95522167029998]
  This article is a self-contained primer on diffusion over general state spaces.<n>We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits.<n>A common variational treatment yields the ELBO that underpins standard training losses.
  arXiv  Detail & Related papers  (2025-12-04T18:55:36Z)
• Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics [13.202844408027412]
  We develop new and sharp convergence guarantees for three popular discrete diffusion models.<n>We show that the computational complexity of each method scales linearly in the dimension, up to logarithmic factors.<n>This study provides the first non-asymptotic convergence guarantees for these noising processes.
  arXiv  Detail & Related papers  (2025-11-29T18:24:43Z)
• A Foundational Theory of Quantitative Abstraction: Adjunctions, Duality, and Logic for Probabilistic Systems [2.362412515574206]
  Large or continuous state spaces make exact analysis intractable and call for principled quantitative abstraction.<n>This work develops a unified theory of such abstraction by integrating category theory, coalgebra, quantitative logic, and optimal transport.
  arXiv  Detail & Related papers  (2025-10-22T10:16:24Z)
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner [66.86440230599656]
  We argue that diffusion language models do not necessarily need to be in the discrete space.<n>In particular, we prove that continuous diffusion models have stronger expressivity than discrete diffusions and looped transformers.<n>We propose Coevolutionary Continuous Diffusion (CCDD), which defines a joint multimodal diffusion process on the union of a continuous representation space and a discrete token space.
  arXiv  Detail & Related papers  (2025-10-03T17:44:41Z)
• Fast-Forward Lattice Boltzmann: Learning Kinetic Behaviour with Physics-Informed Neural Operators [37.65214107289304]
  We introduce a physics-informed neural operator framework for the lattice Boltzmann equation (LBE)<n>Our framework is discretization-invariant, enabling models trained on coarse lattices to generalise to finer ones.<n>Results demonstrate robustness across complex flow scenarios, including von Karman vortex shedding, ligament breakup, and bubble adhesion.
  arXiv  Detail & Related papers  (2025-09-26T14:36:23Z)
• Explicit Quantum Circuits for Simulating Linear Differential Equations via Dilation [0.0]
  We present a concrete pipeline that connects the dilation formalism with explicit quantum circuit constructions.<n>On the analytical side, we introduce a discretization of the continuous dilation operator that is tailored for quantum implementation.<n>We prove that the resulting scheme achieves a global error bound of order $O(M-3/2)$, up to exponentially small boundary effects.
  arXiv  Detail & Related papers  (2025-09-20T18:54:49Z)
• From Linear Differential Equations to Unitaries: A Moment-Matching Dilation Framework with Near-Optimal Quantum Algorithms [0.0]
  We present a universal moment-fulfilling dilation that embeds any linear, non-Hermitian flow into a strictly unitary evolution.<n>We also unveil whole families of new dilations built from differential, integral, pseudo-differential, and difference generators.<n>As concrete demonstrations, we prove that a simple finite-difference dilation in a finite interval attains near-optimal oracle complexity.
  arXiv  Detail & Related papers  (2025-07-14T13:51:38Z)
• Exemplar-Free Continual Learning for State Space Models [32.73275711666184]
  State-Space Models (SSMs) excel at capturing long-range dependencies with structured recurrence.<n>Their evolving internal states pose challenges in adapting them under Continual Learning.<n>We propose Inf-SSM, a novel and simple geometry-aware regularization method.
  arXiv  Detail & Related papers  (2025-05-24T08:59:13Z)
• Tensor network approximation of Koopman operators [0.0]
  We propose a framework for approximating the evolution of observables of measure-preserving ergodic systems. Our approach is based on a spectrally-convergent approximation of the skew-adjoint Koopman generator. A key feature of this quantum-inspired approximation is that it captures information from a tensor product space of dimension $(2d+1)n$.
  arXiv  Detail & Related papers  (2024-07-09T21:40:14Z)
• Exact dynamics of quantum dissipative $XX$ models: Wannier-Stark localization in the fragmented operator space [49.1574468325115]
  We find an exceptional point at a critical dissipation strength that separates oscillating and non-oscillating decay. We also describe a different type of dissipation that leads to a single decay mode in the whole operator subspace.
  arXiv  Detail & Related papers  (2024-05-27T16:11:39Z)
• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.