LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling

• URL: http://arxiv.org/abs/2606.18023v1
• Date: Tue, 16 Jun 2026 15:03:05 GMT
• Title: LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
• Abstract summary: This study studies loop-count selection through a gain--cost view.<n>We instantiate this study by training LoopCoder-v2 with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation.<n> Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tooluse benchmarks.
• Score: 72.71005779366162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Looped Transformers scale latent computation by repeatedly applying shared blocks, but sequential looping increases latency and KV-cache memory with the loop count. Parallel loop Transformers (PLT) alleviate this cost through cross-loop position offsets (CLP) and shared-KV gated sliding-window attention, making loop count a practical design choice. We therefore study PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary. We instantiate this study by training LoopCoder-v2, a family of 7B PLT coders with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation. Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tool-use benchmarks, improving SWE-bench Verified from 43.0 to 64.4 points and Multi-SWE from 14.0 to 31.0 points. In contrast, variants with three or more loops regress, revealing a strongly non-monotonic loop-count effect. Our diagnostics show that loop 2 provides the main productive refinement, while later loops yield diminishing, oscillatory updates and reduced representational diversity. Because the CLP-induced mismatch remains roughly fixed as refinement gains shrink, the offset cost increasingly dominates. This gain--cost trade-off explains PLT's saturation at two loops and provides diagnostics for loop-count selection.

Related papers

• ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding [60.34475274870224]
  ECHO is a hierarchical dual-loop framework that exploits the functional asymmetry between LLM layers.<n>Within the inner loop, early-layer bonus logits drive rapid, multi-step draft-tree exploration at a minimal cost.<n>ECHO significantly boosts mean accepted tokens and achieves a 2.4$times$ to 2.9$times$ speedup.
  arXiv  Detail & Related papers  (2026-09-15T14:21:33Z)
• SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers [50.00050394179417]
  Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size.<n>We study looping on Mixture-of-Experts Transformers while closely matching per-token FLOPs, total non-embedding parameters, and KV cache.<n>Through a series of ablations, we arrive at a recipe we call SMELT, which loops the middle half of layers twice while matching the unlooped Baseline on all three budgets.
  arXiv  Detail & Related papers  (2026-09-01T14:52:57Z)
• Training-Free Hidden-State Refinement for Flow-Matching Image Generators [13.43479682921888]
  We introduce a training-free looping framework that repeatedly applies selected transformer layers inside each denoising call.<n>Across two Scale-RAE model scales, loop variants improve primary and auxiliary quality metrics with competitive quality--efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-08-29T09:17:02Z)
• Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping [56.14767235650558]
  Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length computation tasks.<n>We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops.<n>Our work suggests that "when to stop" should be treated as a training design choice, not merely an inference-time-allocation rule.
  arXiv  Detail & Related papers  (2026-06-29T08:58:09Z)
• Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers [13.254740124473935]
  We introduce LARM, a depth-conditioned looped Transformer that turns recurrent encoder depth into a controllable test-time compute axis.<n>On LibriSpeech, LARM improves WER as the number of inference loops increases.<n>Our results show that test-time compute scaling can extend beyond autoregressive language-model reasoning to continuous non-autoregressive speech recognition.
  arXiv  Detail & Related papers  (2026-06-03T10:01:45Z)
• Simply Stabilizing the Loop via Fully Looped Transformer [41.240805541680395]
  Looped Transformer suffers from training instability when the number of loop iterations increases.<n>Our experiments demonstrate that Fully Looped Transformer improves training stability, enhances downstream performance, and provides preliminary adaptability under different test-time compute budgets.
  arXiv  Detail & Related papers  (2026-05-11T07:21:53Z)
• LoopQ: Quantization for Recursive Transformers [14.001192614503807]
  We present the first systematic study of quantization in LoopLMs.<n>We propose LoopQ, a loop-aware PTQ framework that preserves a shared quantized backbone while introducing lightweight adaptations.<n> Experiments show that, under W4A4 quantization, LoopQ improves average downstream accuracy by 68.8% and reduces average perplexity by 87.7% compared with the strongest static PTQ baseline.
  arXiv  Detail & Related papers  (2026-05-08T01:45:28Z)
• LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction [30.25338458512745]
  Scaling Transformer-based click-through rate (CTR) models by stacking more parameters brings growing computational and storage overhead.<n>We propose LoopCTR, which introduces a loop scaling paradigm that increases training-time computation.<n>Experiments on three public benchmarks and one industrial dataset demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-21T15:06:59Z)
• Parallel Loop Transformer for Efficient Test-Time Computation Scaling [34.8255648599522]
  Large Language Models (LLMs) are powerful but often too slow and costly for real-world use during inference.<n>Looped transformers save on parameters by reusing the same weights for multiple computational steps, or "loops"<n>This approach has a major flaw: the loops run one after another, causing inference latency and memory requirements to increase with each added loop.
  arXiv  Detail & Related papers  (2025-10-28T15:35:50Z)
• On the Constant Depth Implementation of Pauli Exponentials [49.48516314472825]
  We decompose $Zotimes n$ exponentials of arbitrary length into circuits of constant depth using $mathcalO(n)$ ancillae and two-body XX and ZZ interactions.<n>We prove the correctness of our approach, after introducing novel rewrite rules for circuits which benefit from qubit recycling.
  arXiv  Detail & Related papers  (2024-08-15T17:09:08Z)
• Will Bilevel Optimizers Benefit from Loops [63.22466953441521]
  Two current popular bilevelimats AID-BiO and ITD-BiO naturally involve solving one or two sub-problems. We first establish unified convergence analysis for both AID-BiO and ITD-BiO that are applicable to all implementation choices of loops.
  arXiv  Detail & Related papers  (2022-05-27T20:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.