Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers

• URL: http://arxiv.org/abs/2606.04678v1
• Date: Wed, 03 Jun 2026 10:01:45 GMT
• Title: Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers
• Abstract summary: We introduce LARM, a depth-conditioned looped Transformer that turns recurrent encoder depth into a controllable test-time compute axis.<n>On LibriSpeech, LARM improves WER as the number of inference loops increases.<n>Our results show that test-time compute scaling can extend beyond autoregressive language-model reasoning to continuous non-autoregressive speech recognition.
• Score: 13.254740124473935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end ASR systems typically use fixed-depth acoustic encoders at inference, making it difficult to trade additional test-time computation for improved recognition without training a larger model. A natural approach is to reuse a shared Transformer block recurrently, but we find that naive looping does not fully exploit additional recurrent compute. We introduce LARM, a depth-conditioned looped Transformer that turns recurrent encoder depth into a controllable test-time compute axis. LARM combines sparse CTC checkpoints, supervision-clock embeddings, FiLM depth conditioning, and delayed soft-posterior feedback. These components structure the loop into recognition checkpoints separated by latent refinement phases and allow shared weights to specialize across recurrent steps. On LibriSpeech, LARM improves WER as the number of inference loops increases and achieves performance competitive with deeper unshared-parameter baselines. Our results show that test-time compute scaling can extend beyond autoregressive language-model reasoning to continuous non-autoregressive speech recognition.

Related papers

• S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection [74.15663758681849]
  Single-stream framework integrates reliability-calibrated frequency adaptation into the adopted SAM backbone for MSOD.<n>Experiments on RGB-D, RGB-T, and RGB-NIR salient object detection benchmarks validate that the proposed framework achieves competitive performance.
  arXiv  Detail & Related papers  (2026-08-18T08:05:00Z)
• Penelope: Localized Latent Recurrence for Efficient Structured Reasoning [26.549641831847982]
  Penelope is an efficient latent-reasoning framework for pretrained decoder-only Transformers.<n>It localizes recurrent computation to a selected decoder interval.<n>It attains competitive accuracy relative to established latent-reasoning models.
  arXiv  Detail & Related papers  (2026-07-28T16:06:46Z)
• Looped State-Space Language Models with Adaptive Exit-State Selection [43.98434319030707]
  Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than additional independent parameters.<n>We investigate Looped Mamba and Looped Hybrid Mamba-Transformer architectures, which repeatedly apply a shared Mamba block to introduce explicit finite-depth recurrent computation.<n>We adapt Ouro's two-stage exit gate to Looped Mamba for threshold-controlled selection among recurrent-step outputs.
  arXiv  Detail & Related papers  (2026-07-11T04:09:19Z)
• Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping [56.14767235650558]
  Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length computation tasks.<n>We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops.<n>Our work suggests that "when to stop" should be treated as a training design choice, not merely an inference-time-allocation rule.
  arXiv  Detail & Related papers  (2026-06-29T08:58:09Z)
• LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling [72.71005779366162]
  This study studies loop-count selection through a gain--cost view.<n>We instantiate this study by training LoopCoder-v2 with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation.<n> Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tooluse benchmarks.
  arXiv  Detail & Related papers  (2026-06-16T15:03:05Z)
• Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior [107.2098567818173]
  Latent Recurrent Transformer (LRT) is a lightweight augmentation of autoregressive transformers.<n>LRT reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token.
  arXiv  Detail & Related papers  (2026-05-26T10:10:26Z)
• DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation [0.431493144418712]
  We show that feedback loop can induce a previously overlooked failure mode, decoder coupling drift, in which the mask decoder's cross-attention progressively loses alignment with the target object, causing errors to accumulate across iterations.<n>We introduce DeCoDrift, a training-free inference-time stabilization framework that constrains prompt updates and preserves decoder coupling across iterations.
  arXiv  Detail & Related papers  (2026-05-25T11:41:21Z)
• LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models [35.78769938627291]
  We introduce textbfLooped Depth Up-Scaling (LoopUS), a post-training framework that converts a standard pretrained LLM into a looped architecture.<n>LoopUS recasts the pretrained LLM into an encoder, a looped reasoning block, and a decoder.<n>These mechanisms transform a standard non-looped model into a looped form while stabilizing it against both computational bottlenecks and representation collapse.
  arXiv  Detail & Related papers  (2026-05-10T11:05:20Z)
• TempoNet: Slack-Quantized Transformer-Guided Reinforcement Scheduler for Adaptive Deadline-Centric Real-Time Dispatchs [8.818252253980985]
  TempoNet is a reinforcement learning scheduler that pairs a permutation-invariant Transformer with a deep Q-approximation.<n>A latency-aware sparse attention stack with blockwise top-k selection and locality-sensitive chunking enables global reasoning over unordered task sets.
  arXiv  Detail & Related papers  (2026-02-20T09:56:23Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams [63.27233749591346]
  Transformer-based models have dramatically increased their size and parameter count to tackle increasingly complex tasks.<n>Stream data inference is typically performed over a sliding temporal window, leading to highly redundant computations.<n>We propose the Deep Continual Transformer (DeepCoT), a redundancy-free encoder-only model that can be applied over existing deep encoder architectures with minimal changes.
  arXiv  Detail & Related papers  (2025-11-21T16:15:43Z)
• Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation [92.42032403795879]
  We show that pretrained language models (LMs) such as GPT2 still tend to generate repetitive texts. We attribute their overestimation of token-level repetition probabilities to the learning bias. We find that LMs use longer-range dependencies to predict repetitive tokens than non-repetitive ones, which may be the cause of sentence-level repetition loops.
  arXiv  Detail & Related papers  (2023-07-04T07:53:55Z)
• Lightweight and Flexible Deep Equilibrium Learning for CSI Feedback in FDD Massive MIMO [13.856867175477042]
  In frequency-division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems, downlink channel state information (CSI) needs to be sent back to the base station (BS) by the users. We propose a lightweight and flexible deep learning-based CSI feedback approach by capitalizing on deep equilibrium models.
  arXiv  Detail & Related papers  (2022-11-28T05:53:09Z)
• Latent Template Induction with Gumbel-CRFs [107.17408593510372]
  We explore the use of structured variational autoencoders to infer latent templates for sentence generation. As a structured inference network, we show that it learns interpretable templates during training.
  arXiv  Detail & Related papers  (2020-11-29T01:00:57Z)
• Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input [54.82369261350497]
  We propose a CTC-enhanced NAR transformer, which generates target sequence by refining predictions of the CTC module. Experimental results show that our method outperforms all previous NAR counterparts and achieves 50x faster decoding speed than a strong AR baseline with only 0.0 0.3 absolute CER degradation on Aishell-1 and Aishell-2 datasets.
  arXiv  Detail & Related papers  (2020-10-28T15:00:09Z)
• Adapting End-to-End Speech Recognition for Readable Subtitles [15.525314212209562]
  In some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. Experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities.
  arXiv  Detail & Related papers  (2020-05-25T14:42:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.