Descriptive versus Regulatory Uncertainty in Bounded Predictive Systems

• URL: http://arxiv.org/abs/2605.18909v1
• Date: Sun, 17 May 2026 17:41:57 GMT
• Title: Descriptive versus Regulatory Uncertainty in Bounded Predictive Systems
• Abstract summary: We prove that current transformer architectures are confined to descriptive uncertainty at inference.<n> Token-level Shannon entropy is statistically invariant across tasks spanning pattern retrieval, causal operator application, and out-of-distribution causal generalization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Any system that models the world under finite representational capacity must compress; any compression entails a prior; and the prior is the system's bias. What has not been established is whether uncertainty participates in the dynamics governing future behavior, or merely describes the output distribution without consequence. We introduce a structural distinction between descriptive uncertainty, which does not recursively modulate the system's policy, and regulatory uncertainty, which directly enters the optimization landscape and drives persistent adaptive restructuring. We prove formally that current transformer architectures are confined to descriptive uncertainty at inference. We ground this in thermodynamics via Landauer's principle: for uncertainty to be regulatory, epistemic error must cost real energy; in a decoupled system, hallucinations and correct derivations dissipate identical energy. We test this empirically across three locally-deployed language models (3B, 8B, 70B parameters). Token-level Shannon entropy is statistically invariant across tasks spanning pattern retrieval, causal operator application, and out-of-distribution causal generalization in all three models (all pairwise p >= 0.568; within-model ranges 0.011-0.028 nats), while task accuracy varies substantially across the same conditions (0%-100%). Entropy and accuracy are orthogonal. The decoupling is scale-invariant: larger models achieve higher accuracy but identical entropy flatness. This structural incapacity is not resolvable by additional parameters or training data. Genuine epistemic grounding requires physical coupling between thermodynamic substrate state and information processing cost.

Related papers

• Stochastic Counterdiabatic Driving via Biorthogonal Liouvillian Eigenmodes [42.668742604089324]
  Escorted free energy simulations address the non-adiabatic lag by engineering control fields $mathbfu$ that eliminate the lag.<n>We construct a complementary numerical framework based on gauge-type transforms instead of generalized coordinate transforms.<n>We show that the counterdiabatic condition is satisfied to machine precision, with the non-adiabatic lag suppressed by roughly twelve orders of magnitude.
  arXiv  Detail & Related papers  (2026-07-27T13:08:14Z)
• Constructed Reality, Contested Priors: Decoupling and the Architecture of Cognitive Relapse Under the Free Energy Principle [0.0]
  Under the free energy principle, a predictive system does not observe reality directly.<n>It maintains a generative model of the world and experiences that model's best current hypothesis.<n>Can a synthetic environment be made consistent enough that a predictive system's own inference machinery adopts it as this default hypothesis?
  arXiv  Detail & Related papers  (2026-07-12T07:08:49Z)
• Human-Centered Learning Mechanics: A Dynamical Framework for Entropy-Regulated Representation Learning [1.1675763847424785]
  We propose a dynamical and information-theoretic framework for open and controlled learning systems.<n>The central idea is that entropy regularization is useful only when the chosen entropy surrogate generates a non-degenerate information force.<n> Controlled representation-learning experiments support the hypothesis that geometric entropy surrogates induce stronger and more stable information forces than softmax-normalized entropy.
  arXiv  Detail & Related papers  (2026-05-21T18:16:39Z)
• Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates [10.39662017167378]
  Uncertainty quantification (UQ) must be both computationally efficient and spatially faithful.<n>We propose a structure-aware UQ scheme that exploits the modular anatomy common to modern NOs.<n> Experiments on challenging PDE benchmarks demonstrate that the proposed structure-aware design yields more reliable coverage, tighter bands, and improved residual-uncertainty alignment.
  arXiv  Detail & Related papers  (2026-02-24T10:17:24Z)
• Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting [44.23640219583819]
  Reinforced Fine-Tuning (SFT) is the standard paradigm for domain adaptation, yet it frequently incurs the cost of catastrophic forgetting.<n>We propose Entropy-Adaptive Fine-Tuning (EAFT) to solve this problem.<n>EAFT consistently matches the downstream performance of standard SFT while significantly mitigating the degradation of general capabilities.
  arXiv  Detail & Related papers  (2026-01-05T14:28:17Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Gradient Descent as Implicit EM in Distance-Based Neural Models [0.0]
  Neural networks trained with standard objectives exhibit behaviors characteristic of probabilistic inference.<n>We provide a direct derivation to explain these phenomena.<n>No explicit inference algorithm is required because inference is embedded in optimization.
  arXiv  Detail & Related papers  (2025-12-31T10:56:43Z)
• Information-Theoretic Bounds and Task-Centric Learning Complexity for Real-World Dynamic Nonlinear Systems [0.6875312133832079]
  Dynamic nonlinear systems exhibit distortions arising from coupled static and dynamic effects.<n>This paper presents a theoretical framework grounded in structured decomposition, variance analysis, and task-centric complexity bounds.
  arXiv  Detail & Related papers  (2025-09-08T12:08:02Z)
• Energy-based Epistemic Uncertainty for Graph Neural Networks [47.52218144839666]
  We propose an energy-based model (EBM) that provides high-quality uncertainty estimates.<n>We provably induce an integrable density in the data space by regularizing the energy function.<n>Our framework is a simple and effective post hoc method applicable to any pre-trained GNN that is sensitive to various distribution shifts.
  arXiv  Detail & Related papers  (2024-06-06T13:13:29Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting [34.61959169976758]
  APHYNITY is a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists in decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model.
  arXiv  Detail & Related papers  (2020-10-09T09:31:03Z)
• Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions [0.0]
  Statistical uncertainties, coverage, systematic uncertainties or a goodness-of-fit measure are often not calculated. We show that a KL-divergence objective of the joint distribution of data and labels allows to unify supervised learning and variational autoencoders. We discuss how to calculate coverage probabilities without numerical integration for specific "base-ordered" contours.
  arXiv  Detail & Related papers  (2020-08-13T11:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.