LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction

• URL: http://arxiv.org/abs/2604.19550v1
• Date: Tue, 21 Apr 2026 15:06:59 GMT
• Title: LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction
• Abstract summary: Scaling Transformer-based click-through rate (CTR) models by stacking more parameters brings growing computational and storage overhead.<n>We propose LoopCTR, which introduces a loop scaling paradigm that increases training-time computation.<n>Experiments on three public benchmarks and one industrial dataset demonstrate state-of-the-art performance.
• Score: 30.25338458512745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scaling Transformer-based click-through rate (CTR) models by stacking more parameters brings growing computational and storage overhead, creating a widening gap between scaling ambitions and the stringent industrial deployment constraints. We propose LoopCTR, which introduces a loop scaling paradigm that increases training-time computation through recursive reuse of shared model layers, decoupling computation from parameter growth. LoopCTR adopts a sandwich architecture enhanced with Hyper-Connected Residuals and Mixture-of-Experts, and employs process supervision at every loop depth to encode multi-loop benefits into the shared parameters. This enables a train-multi-loop, infer-zero-loop strategy where a single forward pass without any loop already outperforms all baselines. Experiments on three public benchmarks and one industrial dataset demonstrate state-of-the-art performance. Oracle analysis further reveals 0.02--0.04 AUC of untapped headroom, with models trained with fewer loops exhibiting higher oracle ceilings, pointing to a promising frontier for adaptive inference.

Related papers

• SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers [50.00050394179417]
  Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size.<n>We study looping on Mixture-of-Experts Transformers while closely matching per-token FLOPs, total non-embedding parameters, and KV cache.<n>Through a series of ablations, we arrive at a recipe we call SMELT, which loops the middle half of layers twice while matching the unlooped Baseline on all three budgets.
  arXiv  Detail & Related papers  (2026-09-01T14:52:57Z)
• Training-Free Hidden-State Refinement for Flow-Matching Image Generators [13.43479682921888]
  We introduce a training-free looping framework that repeatedly applies selected transformer layers inside each denoising call.<n>Across two Scale-RAE model scales, loop variants improve primary and auxiliary quality metrics with competitive quality--efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-08-29T09:17:02Z)
• LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling [72.71005779366162]
  This study studies loop-count selection through a gain--cost view.<n>We instantiate this study by training LoopCoder-v2 with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation.<n> Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tooluse benchmarks.
  arXiv  Detail & Related papers  (2026-06-16T15:03:05Z)
• Parcae: Scaling Laws For Stable Looped Language Models [35.9547796403241]
  Traditional fixed-depth architectures scale quality by increasing training FLOPs, typically through increased parameterization, at the expense of a higher memory footprint, or data.<n>A potential alternative is looped architectures, which instead increase FLOPs by sending activations through a block of layers in a loop.<n>While promising, existing recipes for training looped architectures can be unstable, suffering from residual explosion and loss spikes.<n>We propose Parcae, a novel stable, looped architecture that constrains the spectral norm of the injection parameters via discretization of a negative diagonal parameterization.
  arXiv  Detail & Related papers  (2026-04-14T16:43:37Z)
• AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization [84.25316984309725]
  Integration of dynamic, sparse structures with parameter-efficient adapters (e.g., LoRA) is a powerful technique for enhancing Large Language Models (LLMs)<n>Despite minimal increases in computational load, the computation latency often skyrockets, leading to decoding speeds slowing by over 2.5 times.<n>AdaFuse is a framework built on a tight co-design between the algorithm and the underlying hardware system to enable efficient dynamic adapter execution.
  arXiv  Detail & Related papers  (2026-03-12T12:46:42Z)
• Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models [63.47909317137073]
  Large Multimodal Models (LMMs) have achieved remarkable success in vision-language computation tasks.<n>But their vast parameter counts are often underutilized during both training and inference.<n>We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs.
  arXiv  Detail & Related papers  (2026-02-09T17:58:23Z)
• LoopViT: Scaling Visual ARC with Looped Transformers [14.9105267508928]
  We propose Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence.<n>Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought.<n> Empirical results on the ARC-AGI-1 benchmark validate this perspective.
  arXiv  Detail & Related papers  (2026-02-02T14:32:57Z)
• DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams [63.27233749591346]
  Transformer-based models have dramatically increased their size and parameter count to tackle increasingly complex tasks.<n>Stream data inference is typically performed over a sliding temporal window, leading to highly redundant computations.<n>We propose the Deep Continual Transformer (DeepCoT), a redundancy-free encoder-only model that can be applied over existing deep encoder architectures with minimal changes.
  arXiv  Detail & Related papers  (2025-11-21T16:15:43Z)
• Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs [61.953548065938385]
  We introduce the ''Three Taxes'' (Bulk Synchronous, Inter- Kernel Data Locality, and Kernel Launch Overhead) as an analytical framework.<n>We propose moving beyond the rigid BSP model to address key inefficiencies in distributed GPU execution.<n>We observe a 10-20% speedup in end-to-end latency over BSP-based approaches.
  arXiv  Detail & Related papers  (2025-11-04T01:15:44Z)
• Parallel Loop Transformer for Efficient Test-Time Computation Scaling [34.8255648599522]
  Large Language Models (LLMs) are powerful but often too slow and costly for real-world use during inference.<n>Looped transformers save on parameters by reusing the same weights for multiple computational steps, or "loops"<n>This approach has a major flaw: the loops run one after another, causing inference latency and memory requirements to increase with each added loop.
  arXiv  Detail & Related papers  (2025-10-28T15:35:50Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• Gradient Coding with Dynamic Clustering for Straggler-Tolerant Distributed Learning [55.052517095437]
  gradient descent (GD) is widely employed to parallelize the learning task by distributing the dataset across multiple workers. A significant performance bottleneck for the per-iteration completion time in distributed synchronous GD is $straggling$ workers. Coded distributed techniques have been introduced recently to mitigate stragglers and to speed up GD iterations by assigning redundant computations to workers. We propose a novel dynamic GC scheme, which assigns redundant data to workers to acquire the flexibility to choose from among a set of possible codes depending on the past straggling behavior.
  arXiv  Detail & Related papers  (2021-03-01T18:51:29Z)
• Gradient Coding with Dynamic Clustering for Straggler Mitigation [57.9123881133818]
  GC-DC regulates the number of straggling workers in each cluster based on the straggler behavior in the previous iteration. We numerically show that GC-DC provides significant improvements in the average completion time (of each iteration) with no increase in the communication load compared to the original GC scheme.
  arXiv  Detail & Related papers  (2020-11-03T18:52:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.