Architectural Proprioception in State Space Models: Thermodynamic Training Induces Anticipatory Halt Detection

• URL: http://arxiv.org/abs/2603.04180v1
• Date: Wed, 04 Mar 2026 15:36:40 GMT
• Title: Architectural Proprioception in State Space Models: Thermodynamic Training Induces Anticipatory Halt Detection
• Abstract summary: We introduce the Probability Navigation Architecture (PNA), which treats computation as navigation through a probability manifold governed by thermodynamic principles.<n>We train State Space Models (SSMs) and Transformers with a novel thermodynamic loss function that penalizes computational waste alongside standard cross-entropy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce the Probability Navigation Architecture (PNA) framework, which treats neural computation as navigation through a probability manifold governed by thermodynamic principles. We train State Space Models (SSMs) and Transformers with a novel thermodynamic loss function that penalizes computational waste alongside standard cross-entropy. Across 19 experimental phases, we discover that thermodynamically-trained SSMs develop architectural proprioception: a strong anticipatory coupling between recurrent state entropy and halt confidence (r = -0.836, p < 0.001) in which the halt signal leads state entropy collapse by exactly two tokens (tau = -2.0). This Universal Stopping Signature (USS) reproduces to four decimal places across random seeds and generalizes to a structurally distinct sorting task. Critically, Transformers trained identically show no such coupling (r = -0.07), demonstrating that the phenomenon is architecture-dependent. Cross-task transfer experiments confirm that SSM halt detection reflects genuine meta-cognition (zero-shot transfer F1: SSMs 64.2% vs. Transformers 69.3%; post-adaptation: SSMs 94.5% vs. Transformers 86.4%), while Transformer halt detection relies on syntactic pattern matching. A 2D hyperparameter sweep over energy penalty (alpha) and halt supervision (beta) reveals that the anticipatory coupling is continuously controllable through training, with thermodynamic pressure serving as the primary induction mechanism and explicit halt supervision as an amplifier. Our results establish that SSMs are thermodynamically native architectures whose fixed-size recurrent states naturally support the Markovian compression that enables computational self-awareness, with implications for cost-aware inference, dynamic token budgets, and confidence-based routing in production systems.

Related papers

• Virtual Temperature Sensors in Power Transformers Using Neural Ordinary Differential Equations [42.50242605797505]
  This paper develops a physics-aware Neural Ordinary Differential Equation framework for forecasting transformer thermal behavior.<n>A key contribution is the integration of simplified heat-transfer equations directly into the Neural ODE formulation.<n>The framework provides a standardized, physics-aware, and robust forecasting approach for heterogeneous transformer units.
  arXiv  Detail & Related papers  (2026-08-13T13:59:32Z)
• Emergence via Phase Transitions: Mechanism Landscapes and Universal Convergence Across Complex Systems [0.0]
  Across machine learning, biology, and physics, independently evolving systems often converge toward strikingly similar high-level structures.<n>We propose the Hierarchical Emergence Framework (HEF) as a candidate universality framework for such convergence phenomena.<n>HEF is not presented as a universal theory of emergence, but as a falsifiable mathematical scaffold for studying convergence phenomena across complex systems.
  arXiv  Detail & Related papers  (2026-05-25T18:32:52Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers [0.0]
  We investigate how the synchronization gap is mechanistically realized within pretrained Diffusion Transformers (DiTs)<n>Our findings provide a mechanistic interpretation of how DiTs resolve generative ambiguity, isolating speciation transitions to the terminal layers of the network.
  arXiv  Detail & Related papers  (2026-03-22T00:13:33Z)
• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift [64.37959940809633]
  We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures.<n>We show Transformer-based sequence policies substantially outperform, RNN, and SSMs in robustness, maintaining high returns even when large fractions of sensors are unavailable.
  arXiv  Detail & Related papers  (2026-03-04T22:21:54Z)
• Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction [45.25461515976432]
  Plug-and-Play diffusion prior (DP) frameworks have emerged as a powerful paradigm for imaging reconstruction.<n>We present a novel approach to resolving bias-hallucination trade-off, achieving state-of-the-art gradients with significantly accelerated convergence.
  arXiv  Detail & Related papers  (2026-02-26T16:58:43Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective [45.713195454899875]
  Diffusion models (DMs) have shown promise for Time-Series Data Imputation.<n>DMs' performance remains inconsistent in complex scenarios.<n>We propose a novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation)
  arXiv  Detail & Related papers  (2026-02-01T12:11:57Z)
• When to Think Fast and Slow? AMOR: Entropy-Based Metacognitive Gate for Dynamic SSM-Attention Switching [4.238040764117957]
  State Space Models (SSMs) offer efficient alternatives but struggle with precise information retrieval over a long horizon.<n>Inspired by dual-process theories of cognition, we propose AMOR, a hybrid architecture that dynamically engages sparse attention only when an SSM backbone is "uncertain"<n>On small-scale synthetic retrieval tasks, AMOR outperforms both SSM-only and transformer-only baselines.
  arXiv  Detail & Related papers  (2026-01-22T17:19:58Z)
• Digital Metabolism: Decoupling Logic from Facts via Regenerative Unlearning -- Towards a Pure Neural Logic Core [4.073707521515039]
  "Digital metabolism" is a hypothesis suggesting that targeted forgetting is necessary for distilling a pure neural logic core.<n>We introduce the Regenerative Logic-Core Protocol (RLCP), a dual-stream training framework that renders specific factual dependencies linearly undecodable.<n> Empirical analysis on GSM8K reveals that the "metabolized" model spontaneously adopts Symbolic chain-of-thought scaffolding.
  arXiv  Detail & Related papers  (2026-01-15T19:21:16Z)
• Block-Recurrent Dynamics in Vision Transformers [42.261020313952976]
  We argue that trained ViTs admit a block-recurrent depth structure such that the computation of the original $L$ blocks can be accurately rewritten using only $k ll L$ distinct blocks applied recurrently.<n>We train a Raptor model to recover $96%$ of DINOv2 ImageNet-1k linear probe accuracy in only 2 blocks at equivalent computational cost.
  arXiv  Detail & Related papers  (2025-12-23T00:18:23Z)
• Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium [0.6820746164515952]
  We introduce the closed-loop prediction principle, which requires that models iteratively refine latent representations until reaching a self-consistent equilibrium.<n>We instantiate this principle as Equilibrium Transformers, which augment standard transformer layers with an Equilibrium Refinement Module.<n>Preliminary experiments on the binary parity task demonstrate +3.28% average improvement on challenging sequences, with gains reaching +8.07% where standard transformers approach random performance.
  arXiv  Detail & Related papers  (2025-11-26T20:02:59Z)
• Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL [56.085103402298905]
  We propose a trajectory entropy-constrained reinforcement learning (TECRL) framework to address these two challenges.<n>Within this framework, we first separately learn two Q-functions, one associated with reward and the other with entropy, ensuring clean and stable value targets unaffected by temperature updates.<n>We develop a practical off-policy algorithm, DSAC-E, by extending the state-of-the-art distributional soft actor-critic with three refinements.
  arXiv  Detail & Related papers  (2025-10-25T09:17:47Z)
• Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking [41.3496135369579]
  Chain-of-thought (CoT) significantly enhances the performance of large language models (LLMs) across a wide range of tasks.<n>There is limited mechanistic understanding of the algorithms that Transformer+CoT can learn.<n>We evaluate the state tracking capabilities of Transformer+CoT and its variants, confirming the effectiveness of CoT.
  arXiv  Detail & Related papers  (2025-02-27T14:24:51Z)
• Anomaly Detection in Complex Dynamical Systems: A Systematic Framework Using Embedding Theory and Physics-Inspired Consistency [0.0]
  Anomaly detection in complex dynamical systems is essential for ensuring reliability, safety, and efficiency in industrial and cyber-physical infrastructures.<n>We propose a system-theoretic approach to anomaly detection, grounded in classical embedding theory and physics-inspired consistency principles.<n>Our findings support the hypothesis that anomalies disrupt stable system dynamics, providing a robust signal for anomaly detection.
  arXiv  Detail & Related papers  (2025-02-26T17:06:13Z)
• State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence [0.0]
  State Stream Transformer (SST) is a novel LLM architecture that reveals emergent reasoning behaviours latent in pretrained weights.<n>SST introduces a sliding window latent state (FFN) cache with weighted decay that maintains and evolves persistent latent processes throughout autoregressive generations.<n>SST achieves substantial performance improvements over the base model on two reasoning benchmarks, reaching 89.01% accuracy on GSM-8K (0-shot) and 91.04% on ARC Challenge (0-shot CoT)
  arXiv  Detail & Related papers  (2025-01-30T14:03:36Z)
• Thermalization and Criticality on an Analog-Digital Quantum Simulator [133.58336306417294]
  We present a quantum simulator comprising 69 superconducting qubits which supports both universal quantum gates and high-fidelity analog evolution. We observe signatures of the classical Kosterlitz-Thouless phase transition, as well as strong deviations from Kibble-Zurek scaling predictions. We digitally prepare the system in pairwise-entangled dimer states and image the transport of energy and vorticity during thermalization.
  arXiv  Detail & Related papers  (2024-05-27T17:40:39Z)
• Probing eigenstate thermalization in quantum simulators via fluctuation-dissipation relations [77.34726150561087]
  The eigenstate thermalization hypothesis (ETH) offers a universal mechanism for the approach to equilibrium of closed quantum many-body systems. Here, we propose a theory-independent route to probe the full ETH in quantum simulators by observing the emergence of fluctuation-dissipation relations. Our work presents a theory-independent way to characterize thermalization in quantum simulators and paves the way to quantum simulate condensed matter pump-probe experiments.
  arXiv  Detail & Related papers  (2020-07-20T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.