Dynamical Systems Theory Behind a Hierarchical Reasoning Model

• URL: http://arxiv.org/abs/2603.22871v1
• Date: Tue, 24 Mar 2026 07:14:22 GMT
• Title: Dynamical Systems Theory Behind a Hierarchical Reasoning Model
• Abstract summary: Contraction Mapping Model (CMM) is a novel architecture that reformulates discrete reasoning into continuous Neural Ordinary and Differential Equations (NODEs/NSDEs)<n>CMM achieves a state-of-the-art accuracy of 93.7 %, outperforming the 27M-Extreme benchmark and 85.4 % on Sudoku-Extreme benchmark.<n>Results establish a new frontier for extreme parameter efficiency, proving that mathematically rigorous latent dynamics can effectively replace brute-force scaling in artificial reasoning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current large language models (LLMs) primarily rely on linear sequence generation and massive parameter counts, yet they severely struggle with complex algorithmic reasoning. While recent reasoning architectures, such as the Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), demonstrate that compact recursive networks can tackle these tasks, their training dynamics often lack rigorous mathematical guarantees, leading to instability and representational collapse. We propose the Contraction Mapping Model (CMM), a novel architecture that reformulates discrete recursive reasoning into continuous Neural Ordinary and Stochastic Differential Equations (NODEs/NSDEs). By explicitly enforcing the convergence of the latent phase point to a stable equilibrium state and mitigating feature collapse with a hyperspherical repulsion loss, the CMM provides a mathematically grounded and highly stable reasoning engine. On the Sudoku-Extreme benchmark, a 5M-parameter CMM achieves a state-of-the-art accuracy of 93.7 %, outperforming the 27M-parameter HRM (55.0 %) and 5M-parameter TRM (87.4 %). Remarkably, even when aggressively compressed to an ultra-tiny footprint of just 0.26M parameters, the CMM retains robust predictive power, achieving 85.4 % on Sudoku-Extreme and 82.2 % on the Maze benchmark. These results establish a new frontier for extreme parameter efficiency, proving that mathematically rigorous latent dynamics can effectively replace brute-force scaling in artificial reasoning.

Related papers

• Yield Curves Dynamics Using Variational Autoencoders Under No-arbitrage [0.0]
  This paper introduces a physics-informed framework that resolves the conflict between the statistical flexibility of deep learning and the rigorous theoretical constraints of fixed-income modeling.<n>We demonstrate that standard generative models and unconstrained statistical extrapolations suffer from "manifold collapse" and severe arbitrage violations when forecasting term structures across diverse macroeconomic regimes.<n>Ultimately, this research provides a highly scalable, mathematically sound evolutionary engine for term structure modeling.
  arXiv  Detail & Related papers  (2026-05-12T21:21:25Z)
• Robust Reasoning Benchmark [2.622879924644665]
  We propose a perturbation pipeline consisting of 14 techniques to evaluate robustness of LLM reasoning.<n>We apply this pipeline to AIME 2024 dataset and evalute 8 state-of-the-art models on the resulting benchmark.<n>Our results indicate that open weight models ranging from 7B to 120B parameters and Claude Opus 4.6 exhibit accuracy decay on subsequent problems.
  arXiv  Detail & Related papers  (2026-03-26T22:19:33Z)
• Hybrid Hidden Markov Model for Modeling Equity Excess Growth Rate Dynamics: A Discrete-State Approach with Jump-Diffusion [0.0]
  Existing approaches, from parametric models to deep generative networks, struggle to simultaneously reproduce heavy-tailed distributions, negligible linear autocorrelation, and persistent volatility clustering.<n>We propose a hybrid hidden Markov framework that discretizes continuous excess growth rates into Laplace quantile-defined market states.<n> Parameters are estimated by direct transition counting, bypassing the Baum-Welch EM algorithm.
  arXiv  Detail & Related papers  (2026-03-10T20:06:53Z)
• Tiny Autoregressive Recursive Models [48.902738432071935]
  Tiny Recursive Models (TRMs) have recently demonstrated remarkable performance on ARC-AGI.<n>We propose the Autoregressive TRM and evaluate it on small autoregressive tasks.<n>Contrary to expectations, we find no reliable performance gains from the full Autoregressive TRM architecture.
  arXiv  Detail & Related papers  (2026-03-09T08:22:45Z)
• CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization [5.857877898558651]
  Chain-of-Thought (CoT) reasoning enhances the problem-solving ability of large language models (LLMs) but leads to substantial inference overhead.<n>This paper investigates efficient CoT transfer across models of different scales and architectures through an adaptive reasoning summarization framework.
  arXiv  Detail & Related papers  (2025-11-07T22:35:31Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• Robust Iterative Learning Hidden Quantum Markov Models [0.7493761475572844]
  Hidden Quantum Markov Models (HQMMs) extend classical Hidden Markov Models to the quantum domain.<n>Existing HQMM learning algorithms are sensitive to data corruption and lack mechanisms to ensure robustness under adversarial perturbations.<n>We introduce the Adversarially Corrupted HQMM, which formalizes robustness analysis by allowing a controlled fraction of observation sequences to be adversarially corrupted.
  arXiv  Detail & Related papers  (2025-10-27T11:48:44Z)
• CALM Before the STORM: Unlocking Native Reasoning for Optimization Modeling [60.55856973678002]
  Large Reasoning Models (LRMs) have demonstrated strong capabilities in complex multi-step reasoning.<n>Existing domain adaptation methods, originally designed for earlier instruction-tuned models, often fail to exploit the advanced reasoning patterns of modern LRMs.<n>We propose textbfCALM, a framework that progressively refines LRMs within their native reasoning modes for optimization modeling tasks.
  arXiv  Detail & Related papers  (2025-10-05T13:38:31Z)
• From Static to Dynamic: Adaptive Monte Carlo Search for Mathematical Process Supervision [49.59309446816251]
  Existing methods estimate the quality of reasoning steps based on a fixed-budget sampling strategy.<n>We propose Adaptive Monte Carlo Search (AMCS), a framework that transforms data generation from fixed, static to adaptive.<n>AMCS adaptively refines estimation by allocating more samples to uncertain reasoning steps while using fewer samples for those that are easier to estimate.
  arXiv  Detail & Related papers  (2025-09-29T06:52:35Z)
• Transition Models: Rethinking the Generative Learning Objective [68.16330673177207]
  We introduce a continuous-time dynamics equation that analytically defines state transitions across any finite time interval.<n>This leads to a novel generative paradigm, Transition Models (TiM), which adapt to arbitrary-step transitions.<n>TiM achieves state-of-the-art performance, surpassing leading models such as SD3.5 (8B parameters) and FLUX.1 (12B parameters) across all evaluated step counts.
  arXiv  Detail & Related papers  (2025-09-04T17:05:59Z)
• HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware [4.727184737671133]
  State Space Models (SSMs) are efficient alternatives to traditional sequence models.<n>Their reliance on matrix multiplications makes them ideal for compute-in-memory (CIM) architectures.<n>We propose HPD, a Hybrid Projection Decomposition strategy for the last output projection layer.
  arXiv  Detail & Related papers  (2025-08-16T06:34:14Z)
• Data-free Weight Compress and Denoise for Large Language Models [96.68582094536032]
  We propose a novel approach termed Data-free Joint Rank-k Approximation for compressing the parameter matrices.<n>We achieve a model pruning of 80% parameters while retaining 93.43% of the original performance without any calibration data.
  arXiv  Detail & Related papers  (2024-02-26T05:51:47Z)
• Efficient hierarchical Bayesian inference for spatio-temporal regression models in neuroimaging [6.512092052306553]
  Examples include M/EEG inverse problems, encoding neural models for task-based fMRI analyses, and temperature monitoring schemes. We devise a novel hierarchical flexible Bayesian framework within which the intrinsic-temporal dynamics of model parameters and noise are modeled.
  arXiv  Detail & Related papers  (2021-11-02T15:50:01Z)
• Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling [76.89779231460193]
  factorization machine (FM) is capable of automatically learning high-order interactions among features to make predictions without the need for manual feature engineering. We propose the quaternion factorization machine (QFM) and quaternion neural factorization machine (QNFM) for sparse predictive analytics.
  arXiv  Detail & Related papers  (2021-04-05T00:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.