Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling

• URL: http://arxiv.org/abs/2601.02346v1
• Date: Mon, 05 Jan 2026 18:44:27 GMT
• Title: Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
• Abstract summary: Falcon-H1R consistently matches or outperforming SOTA reasoning models $2times$ to $7times$ larger.<n>Falcon-H1R advances the 3D limits of reasoning efficiency by combining faster inference, token efficiency, and higher accuracy.<n>Falcon-H1R demonstrates that compact models, through targeted model training and architectural choices, can deliver robust and scalable reasoning performance.
• Score: 7.924613201525482
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work introduces Falcon-H1R, a 7B-parameter reasoning-optimized model that establishes the feasibility of achieving competitive reasoning performance with small language models (SLMs). Falcon-H1R stands out for its parameter efficiency, consistently matching or outperforming SOTA reasoning models that are $2\times$ to $7\times$ larger across a variety of reasoning-intensive benchmarks. These results underscore the importance of careful data curation and targeted training strategies (via both efficient SFT and RL scaling) in delivering significant performance gains without increasing model size. Furthermore, Falcon-H1R advances the 3D limits of reasoning efficiency by combining faster inference (through its hybrid-parallel architecture design), token efficiency, and higher accuracy. This unique blend makes Falcon-H1R-7B a practical backbone for scaling advanced reasoning systems, particularly in scenarios requiring extensive chain-of-thoughts generation and parallel test-time scaling. Leveraging the recently introduced DeepConf approach, Falcon-H1R achieves state-of-the-art test-time scaling efficiency, offering substantial improvements in both accuracy and computational cost. As a result, Falcon-H1R demonstrates that compact models, through targeted model training and architectural choices, can deliver robust and scalable reasoning performance.

Related papers

• Fast-Slow Thinking RM: Efficient Integration of Scalar and Generative Reward Models [16.460841602259787]
  We introduce Fast-Slow Thinking Reward Models (F/S-RM), a hybrid RM architecture inspired by Dual Process Theory.<n>It trains a single model to integrate two distinct reward paradigms: first-token prediction as a scalar score (fast thinking) and CoT-based judgment (slow thinking)<n>F/S-RM achieves a 1.2% relative performance improvement over state-of-the-art models while reducing token consumption by 20.8%.
  arXiv  Detail & Related papers  (2026-03-02T15:48:54Z)
• MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling [80.48332380100915]
  MiniCPM-SALA is a hybrid model that integrates the high-fidelity long-context modeling of sparse attention with the global efficiency of linear attention.<n>On a single NVIDIA A6000D GPU, the model achieves up to 3.5x the inference speed of the full-attention model at the sequence length of 256K tokens.
  arXiv  Detail & Related papers  (2026-02-12T09:37:05Z)
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models [99.6720868215076]
  We introduce ThreadWeaver, a framework for adaptive parallel reasoning.<n> ThreadWeaver achieves accuracy on par with popular sequential reasoning models of comparable size.<n>We show that ThreadWeaver delivers up to 1.53x average speedup in token latency.
  arXiv  Detail & Related papers  (2025-11-24T18:55:59Z)
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail [85.47497935739936]
  Alpamayo-R1 (AR1) is a vision-language-action model that integrates Chain of Causation reasoning with trajectory planning.<n>We show AR1 achieves 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline.<n>We plan to release AR1 models and a subset of the CoC in a future update.
  arXiv  Detail & Related papers  (2025-10-30T01:25:34Z)
• Efficient Large Language Models with Zero-Shot Adjustable Acceleration [4.125187280299246]
  This paper introduces Zero-Shot Adjustable Acceleration, a novel training and inference method that adjusts dynamically hardware utilization during inference without requiring additional fine-tuning.<n> Experimental results demonstrate that the method supports a wide range of zero-shot acceleration and achieves up to 11x speedup compared to the baseline.
  arXiv  Detail & Related papers  (2025-09-01T07:15:25Z)
• Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance [7.261605702995345]
  Falcon-H1 is a new series of large language models (LLMs) featuring hybrid architecture designs optimized for both high performance and efficiency.<n>Falcon-H1 is released in multiple configurations, including base and instruction-tuned variants at 0.5B, 1.5B, 1.5B-deep, 3B, 7B, and 34B parameters.<n>With support for up to 256K context tokens and 18 languages, Falcon-H1 is suitable for a wide range of applications.
  arXiv  Detail & Related papers  (2025-07-30T07:55:33Z)
• ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model [7.798551697095774]
  ReasoningV is a novel model that integrates trained intrinsic capabilities with dynamic inference adaptation for Verilog code generation.<n>Our framework introduces three complementary innovations: ReasoningV-5K, a high-quality dataset of 5,000 functionally verified instances with reasoning paths created through multi-dimensional filtering of PyraNet samples.<n> Experimental results demonstrate ReasoningV's effectiveness with a pass@1 accuracy of 57.8% on VerilogEval-human.
  arXiv  Detail & Related papers  (2025-04-20T10:16:59Z)
• DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models [50.32663816994459]
  Diffusion-styled Preference Optimization (model) provides an efficient and policy-agnostic solution for aligning LLMs with humans.<n>modelavoids the time latency associated with token-level generation.<n>Experiments on AlpacaEval 2, MT-bench, and HH-RLHF demonstrate that modelachieves superior alignment performance across various settings.
  arXiv  Detail & Related papers  (2025-03-06T09:21:54Z)
• Building Efficient Lightweight CNN Models [0.0]
  Convolutional Neural Networks (CNNs) are pivotal in image classification tasks due to their robust feature extraction capabilities.<n>This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy.<n>The proposed model achieved a state-of-the-art accuracy of 99% on the handwritten digit MNIST and 89% on fashion MNIST, with only 14,862 parameters and a model size of 0.17 MB.
  arXiv  Detail & Related papers  (2025-01-26T14:39:01Z)
• Hymba: A Hybrid-head Architecture for Small Language Models [65.94140459055244]
  Hymba is a family of small language models featuring a hybrid-head parallel architecture. We introduce learnable meta tokens that are prepended to prompts, storing critical information. This model is further optimized by incorporating cross-layer key-value sharing and partial sliding window attention.
  arXiv  Detail & Related papers  (2024-11-20T19:51:25Z)
• Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models [54.02863371927658]
  Large Language Models (LLMs) have become indispensable in numerous real-world applications.<n>Ferret is the first first-order method with shared randomness to enable scalable full- parameter tuning of LLMs.<n>Ferret achieves high computational efficiency, reduced communication overhead, and fast convergence.
  arXiv  Detail & Related papers  (2024-09-10T07:28:13Z)
• FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completion [24.964973946366335]
  We develop a novel retrieval-based method, FT2Ra, which aims to mimic genuine fine-tuning. FT2Ra achieves a 4.29% improvement in accuracy compared to the best baseline method on UniXcoder.
  arXiv  Detail & Related papers  (2024-04-02T01:42:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.