T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning With Dynamic Routing

• URL: http://arxiv.org/abs/2609.15160v1
• Date: Mon, 14 Sep 2026 07:41:37 GMT
• Title: T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning With Dynamic Routing
• Abstract summary: Looped Transformers have recently demonstrated strong performance in both reasoning and language tasks.<n>In this work, we propose textbfdynamic token-choice routing for looped transformers.<n>We use a dynamic router to decide whether a token should continue recursing or exit early, allowing simple tokens to bypass unnecessary computation.
• Score: 31.346695540393657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Looped Transformers have recently demonstrated strong performance in both reasoning and language tasks by reusing a shared set of parameters across multiple iterations, achieving parameter efficiency without sacrificing representational power. Besides, looped Transformers perform inference directly in the latent space (latent reasoning) to reduce the number of tokens consumed during inference, thereby achieving improved sample efficiency. However, these models typically apply a fixed recursion depth uniformly to every token, leading to suboptimal compute allocation and leaving significant efficiency gains on the table. In this work, we propose \textbf{dynamic token-choice routing} for looped transformers, enabling each token to adaptively determine its own number of loop iterations based on its hidden state. We use a dynamic router to decide whether a token should continue recursing or exit early, allowing simple tokens to bypass unnecessary computation while hard tokens receive deeper processing. To ensure that this adaptive mechanism does not compromise decoding efficiency, we further introduce recursion-wise KV caching, which maintains an independent key-value cache for each recursion loop. This design ensures that tokens at different depths only attend to their corresponding cached states, effectively eliminating redundant computations for exited tokens and enabling fast autoregressive decoding. Extensive experiments show that T-LoopFormer reaches the sota performance under the same parameters on PPL and 10 zero-shot reasoning tasks, even surpassing the base model at 24x FLOPs and our model could reach the lowest inference latency, which validate the effectiveness of token-choice router and recursion-wise KV cache. Code: https://github.com/YuMingQian1234/T-LoopFormer.

Related papers

• LoopMTP: A looped transformer guided by latent multi-token prediction [2.2694459418593698]
  Multi-token prediction (MTP) supplies exactly the dense, forward-looking supervision the loop is missing.<n>textscLoopMTP improves average accuracy by up to 8.1% (relative) over the non-looped baseline.
  arXiv  Detail & Related papers  (2026-08-04T13:13:09Z)
• Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior [107.2098567818173]
  Latent Recurrent Transformer (LRT) is a lightweight augmentation of autoregressive transformers.<n>LRT reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token.
  arXiv  Detail & Related papers  (2026-05-26T10:10:26Z)
• Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees [68.04613115686509]
  Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting.<n>We propose Distinct Leafion (DLE), a deterministic decoding method that treats truncated sampling as a pruned tree and systematically enumerates distinct leaves instead of sampling with replacement.<n>DLE explores higher-quality reasoning traces, yielding better performance on math, coding, and general reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-22T12:42:03Z)
• Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models [63.47909317137073]
  Large Multimodal Models (LMMs) have achieved remarkable success in vision-language computation tasks.<n>But their vast parameter counts are often underutilized during both training and inference.<n>We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs.
  arXiv  Detail & Related papers  (2026-02-09T17:58:23Z)
• DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers [28.595962720945348]
  Transformers achieve state-of-the-art results across many tasks, but their uniform application of quadratic self-attention makes them computationally expensive.<n>We introduce Dynamic Token Routing Network, an improved Transformer architecture that allows tokens to dynamically skip the quadratic cost of cross-token mixing.
  arXiv  Detail & Related papers  (2025-08-31T16:21:21Z)
• Change of Thought: Adaptive Test-Time Computation [0.0]
  Self-Transformers recover much of the expressive power of iterative reasoning.<n>Self-Transformers iteratively refine their attention weights to a fixed point.
  arXiv  Detail & Related papers  (2025-07-17T23:12:57Z)
• Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation [61.67090981767583]
  We introduce Mixture-of-Recursions (MoR), a unified framework that combines the two axes of efficiency inside a single Recursive Transformer.<n>MoR reuses a shared stack of layers across recursion steps to achieve parameter efficiency, while lightweight routers enable adaptive token-level thinking.<n>We also propose a KV sharing variant that reuses KV pairs from the first recursion, specifically designed to further decrease memory footprint.
  arXiv  Detail & Related papers  (2025-07-14T17:49:00Z)
• One-Pass to Reason: Token Duplication and Block-Sparse Mask for Efficient Fine-Tuning on Multi-Turn Reasoning [13.831457888508892]
  Fine-tuning Large Language Models (LLMs) on multi-turn reasoning datasets requires N (number of turns) separate forward passes per conversation.<n>We propose duplicating response tokens along with a custom attention mask to enable single-pass processing of entire conversations.
  arXiv  Detail & Related papers  (2025-04-25T10:46:56Z)
• TORE: Token Recycling in Vision Transformers for Efficient Active Visual Exploration [2.177039289023855]
  Active Visual Exploration (AVE) optimize the utilization of robotic resources in real-world scenarios by sequentially selecting the most informative observations. We introduce a novel approach to AVE called TOken REcycling (TORE) It divides the encoder into extractor and aggregator components. The extractor processes each observation separately, enabling the reuse of tokens passed to the aggregator.
  arXiv  Detail & Related papers  (2023-11-26T15:39:57Z)
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition [62.83832841523525]
  We propose a fast and accurate parallel transformer, termed Paraformer. It accurately predicts the number of output tokens and extract hidden variables. It can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.
  arXiv  Detail & Related papers  (2022-06-16T17:24:14Z)
• Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing [112.2208052057002]
  We propose Funnel-Transformer which gradually compresses the sequence of hidden states to a shorter one. With comparable or fewer FLOPs, Funnel-Transformer outperforms the standard Transformer on a wide variety of sequence-level prediction tasks.
  arXiv  Detail & Related papers  (2020-06-05T05:16:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.