CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models

• URL: http://arxiv.org/abs/2605.28919v1
• Date: Wed, 27 May 2026 17:59:14 GMT
• Title: CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models
• Abstract summary: We present CosmicFish-HRM, a compact language model with adaptive reasoning depth.<n>The model learns non-uniform reasoning behavior, allocating different numbers of reasoning steps across tasks and inputs.<n>These findings suggest that adaptive reasoning depth may offer a promising alternative to relying solely on parameter scale for reasoning capability.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models have achieved strong reasoning capabilities, though often at the cost of massive parameter counts and expensive inference. In this work, we explore a different direction: adaptive reasoning depth in compact language models. We present CosmicFish-HRM, a compact language model built around a Hierarchical Reasoning Module (HRM) that dynamically allocates computational effort during inference. Instead of applying fixed computation to every input, the model iterates through high-level and low-level reasoning cycles and learns when to halt based on input complexity. CosmicFish-HRM combines this adaptive reasoning core with modern transformer components including Grouped Query Attention, RoPE, and SwiGLU activations. While the additional reasoning infrastructure introduces overhead at small scale, we hypothesize that this tradeoff becomes increasingly favorable as model size grows and the relative cost of the HRM core diminishes. Our results show that the model learns non-uniform reasoning behavior, allocating different numbers of reasoning steps across tasks and inputs. These findings suggest that adaptive reasoning depth may offer a promising alternative to relying solely on parameter scale for reasoning capability.

Related papers

• Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models [47.195670444638715]
  We identify and formally define textbfEntropy-Gradient Inversion, a robust correlation between token entropy and logit gradients.<n>We propose textbfCorrelation-Regularized Group Policy Optimization (CorR-PO), which embeds this inversion signature into reinforcement learning.<n>Experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-18T02:41:53Z)
• CiPO: Counterfactual Unlearning for Large Reasoning Models through Iterative Preference Optimization [54.774620283208776]
  Large Reasoning Models (LRMs) emphasize long chain-of-thought (CoT) reasoning to address complex questions.<n>Existing methods either struggle to completely eliminate undesired knowledge from the CoT traces or degrade the reasoning performances due to the interference with the reasoning process.<n>We introduce Counterfactual Unlearning through iterative Preference Optimization (CiPO), a novel framework that redefines unlearning as the targeted intervention of the CoT reasoning in LRMs.
  arXiv  Detail & Related papers  (2026-04-17T08:56:36Z)
• Decoding the Critique Mechanism in Large Reasoning Models [50.821607345799386]
  Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions.<n>We investigate how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps.<n>We find that despite the error propagating through the chain-of-thought, the model still reaches the correct final answer.
  arXiv  Detail & Related papers  (2026-03-17T10:03:30Z)
• To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks [56.11584171938381]
  Theory of Mind (ToM) assesses whether models can infer hidden mental states such as beliefs, desires, and intentions.<n>Recent progress in Large Reasoning Models (LRMs) has boosted step-by-step inference in mathematics and coding.<n>We present a systematic study of nine advanced Large Language Models (LLMs) comparing reasoning models with non-reasoning models.
  arXiv  Detail & Related papers  (2026-02-11T08:16:13Z)
• Reasoning Pattern Alignment Merging for Adaptive Reasoning [48.347817456299104]
  Reasoning Pattern Alignment Merging (RPAM)<n>RPAM is a layer-wise model merging framework based on feature alignment to facilitate query-adaptive reasoning.<n> Experiments on seven widely used reasoning benchmarks show that RPAM substantially reduces inference cost while maintaining strong performance.
  arXiv  Detail & Related papers  (2026-01-07T01:36:39Z)
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs [49.66344956133349]
  Reasoning capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models.<n>This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a latent variable for strategic contextualization.
  arXiv  Detail & Related papers  (2025-12-19T03:32:53Z)
• Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modeling [23.919163488129985]
  The Structural Reward Model (SRM) is a modular framework integrating side-branch and auxiliary feature generators.<n>By introducing fine-grained dimensions, RMs enable interpretable and efficient evaluations, targeted diagnostics and optimization.
  arXiv  Detail & Related papers  (2025-09-29T18:09:25Z)
• AdapThink: Adaptive Thinking Preferences for Reasoning Language Model [32.47427081297578]
  Reinforcement Learning (RL)-based post-training has significantly advanced the complex reasoning capabilities of language models.<n>However, this slow thinking'' paradigm presents a critical challenge to reasoning efficiency.<n>We propose AdapThink, an adaptive post-training framework designed to induce more efficient thinking.
  arXiv  Detail & Related papers  (2025-06-23T02:06:04Z)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity [16.266145641151375]
  Large Reasoning Models generate detailed thinking processes before providing answers.<n>We show that LRMs face a complete accuracy collapse beyond certain complexities.<n>We also investigate the reasoning traces in more depth, studying the patterns of explored solutions.
  arXiv  Detail & Related papers  (2025-06-07T22:42:29Z)
• Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism [68.05754701230039]
  We construct a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models.<n>We propose a random matrix-based algorithm to enhance the model's reasoning ability.
  arXiv  Detail & Related papers  (2024-05-24T07:41:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.