RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance

• URL: http://arxiv.org/abs/2105.08820v1
• Date: Tue, 18 May 2021 20:44:04 GMT
• Title: RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance
• Abstract summary: RecPipe is a system to jointly optimize recommendation quality and inference performance. RPAccel is a custom accelerator that jointly optimize quality, tail-latency, and system throughput.
• Score: 6.489720534548981
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning recommendation systems must provide high quality, personalized content under strict tail-latency targets and high system loads. This paper presents RecPipe, a system to jointly optimize recommendation quality and inference performance. Central to RecPipe is decomposing recommendation models into multi-stage pipelines to maintain quality while reducing compute complexity and exposing distinct parallelism opportunities. RecPipe implements an inference scheduler to map multi-stage recommendation engines onto commodity, heterogeneous platforms (e.g., CPUs, GPUs).While the hardware-aware scheduling improves ranking efficiency, the commodity platforms suffer from many limitations requiring specialized hardware. Thus, we design RecPipeAccel (RPAccel), a custom accelerator that jointly optimizes quality, tail-latency, and system throughput. RPAc-cel is designed specifically to exploit the distinct design space opened via RecPipe. In particular, RPAccel processes queries in sub-batches to pipeline recommendation stages, implements dual static and dynamic embedding caches, a set of top-k filtering units, and a reconfigurable systolic array. Com-pared to prior-art and at iso-quality, we demonstrate that RPAccel improves latency and throughput by 3x and 6x.

Related papers

• ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation [43.781215072800364]
  Request-Oriented Compute Sharing (ROCS) exploits a unique property of recommendation inference.<n>ROCS defers request-candidate interactions as late as possible, isolates candidate-dependent representations, and evaluates substantial portions of the model once per request.<n>ROCS has been deployed across large-scale recommendation systems spanning ads and organic surfaces, retrieval and ranking stages, and more than two orders of magnitude in inference complexity.
  arXiv  Detail & Related papers  (2026-07-30T06:33:50Z)
• VQA for Dynamic Portfolio Optimization: Sampling Strategies, Optimizer Scheduling, and Hardware-Aware Ansatz Design [0.0]
  Variational quantum algorithms are increasingly explored for optimization problems at scales relevant to near-term quantum devices.<n>Their practical performance depends strongly on design choices such as the objective, classical, and ansatz layout before and after hardware transpilation.<n>We study these factors for dynamic portfolio optimization, a multi-period financial problem balancing return, risk, transaction costs, cash-interest effects, and constraints.
  arXiv  Detail & Related papers  (2026-06-08T19:25:39Z)
• ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting [65.02700555625749]
  ForeSplat is an optimization-aware training framework for feed-forward 3DGS models.<n>ForeSplat substantially reduces the capacity pressure on the feed-forward model.<n>It bridges the gap between amortized prediction and per-scene optimization.
  arXiv  Detail & Related papers  (2026-05-21T05:38:47Z)
• Learn2Splat: Extending the Horizon of Learned 3DGS Optimization [53.913063313338874]
  We introduce a learned for 3DGS that avoids degradation over extended horizons without auxiliary mechanisms.<n>Results improved novel view quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings.
  arXiv  Detail & Related papers  (2026-05-15T09:21:29Z)
• AutoScout: Structured Optimization for Automating ML System Configuration [12.657466158068972]
  AutoScout is a general-purpose system for machine learning training, fine-tuning, and inference.<n>It prioritizes high-impact configuration features and ensembles simulators with varying fidelity.<n>It consistently identifies high-performance configurations, achieving 2.7-3.0$times$ training speedup over expert-tuned settings.
  arXiv  Detail & Related papers  (2026-03-12T06:56:12Z)
• EPOCH: An Agentic Protocol for Multi-Round System Optimization [0.4269415289712572]
  We introduce EPOCH, an engineering protocol for multi-round system optimization in heterogeneous environments.<n>EPOCH organizes optimization into two phases: baseline construction and iterative self-improvement.<n>It further structures each round through role-constrained stages that separate planning, implementation, and evaluation.
  arXiv  Detail & Related papers  (2026-03-10T00:41:03Z)
• SCASRec: A Self-Correcting and Auto-Stopping Model for Generative Route List Recommendation [21.36279838596394]
  Route recommendation systems commonly adopt a multi-stage pipeline involving fine-ranking and re-ranking to produce high-quality ordered recommendations.<n>We propose SCASRec, a unified generative framework that integrates ranking and redundancy elimination into a single end-to-end process.<n> SCASRec introduces a stepwise corrective reward (SCR) to guide list-wise refinement by focusing on hard samples, and employs a learnable End-of-Recommendation (EOR) token to terminate generation adaptively when no further improvement is expected.
  arXiv  Detail & Related papers  (2026-02-03T09:51:58Z)
• AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving [16.664502126572856]
  AIConfigurator is a unified performance-modeling system for Large Language Model (LLM) inference.<n>It enables rapid, framework-a configuration search without requiring GPU-based profiling.<n>It identifies superior serving configurations that improve performance by up to 40% for dense models.
  arXiv  Detail & Related papers  (2026-01-09T20:03:57Z)
• Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards [95.19837878559456]
  We propose Optimas, a unified framework for effective optimization of compound systems.<n>In each iteration, Optimas efficiently adapts the Local Reward Function (LRF) to maintain this property while simultaneously maximizing each component's local reward.<n>We present extensive evaluations across five real-world compound systems to demonstrate that Optimas outperforms strong baselines by an average improvement of 11.92%.
  arXiv  Detail & Related papers  (2025-07-03T07:12:48Z)
• Deep Learning Model Acceleration and Optimization Strategies for Real-Time Recommendation Systems [1.9316786310787222]
  Key challenge for real-time recommendation systems is how to reduce inference latency and increase system throughput without sacrificing recommendation quality.<n>This paper proposes a combined set of modeling- and system-level acceleration and optimization strategies.<n> Experiments show that, while maintaining the original recommendation accuracy, our methods cut latency to less than 30% of the baseline and more than double system throughput.
  arXiv  Detail & Related papers  (2025-06-13T02:39:21Z)
• AI-Driven Optimization of Hardware Overlay Configurations [0.0]
  This paper presents an AI-driven approach to optimizing FPGA overlay configurations. By leveraging machine learning techniques, we predict the feasibility and efficiency of different configurations before hardware compilation.
  arXiv  Detail & Related papers  (2025-03-08T22:34:47Z)
• HEPPO: Hardware-Efficient Proximal Policy Optimization -- A Universal Pipelined Architecture for Generalized Advantage Estimation [0.0]
  HEPPO is an FPGA-based accelerator designed to optimize the Generalized Advantage Estimation stage in Proximal Policy Optimization. Key innovation is our strategic standardization technique, which combines dynamic reward standardization and block standardization for values, followed by 8-bit uniform quantization. Our single-chip solution minimizes communication latency and throughput bottlenecks, significantly boosting PPO training efficiency.
  arXiv  Detail & Related papers  (2025-01-22T08:18:56Z)
• Towards Automated Model Design on Recommender Systems [21.421326082345136]
  We introduce a novel paradigm that utilizes weight sharing to explore abundant solution spaces. From a co-design perspective, we achieve 2x FLOPs efficiency, 1.8x energy efficiency, and 1.5x performance improvements in recommender models.
  arXiv  Detail & Related papers  (2024-11-12T06:03:47Z)
• Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design [59.00758127310582]
  We propose a novel framework Read-ME that transforms pre-trained dense LLMs into smaller MoE models. Our approach employs activation sparsity to extract experts. Read-ME outperforms other popular open-source dense models of similar scales.
  arXiv  Detail & Related papers  (2024-10-24T19:48:51Z)
• DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction [57.83978915843095]
  This paper introduces DiSK, a novel framework designed to significantly enhance the performance of differentially private gradients. To ensure practicality for large-scale training, we simplify the Kalman filtering process, minimizing its memory and computational demands.
  arXiv  Detail & Related papers  (2024-10-04T19:30:39Z)
• Generative Recommender with End-to-End Learnable Item Tokenization [51.82768744368208]
  We introduce ETEGRec, a novel End-To-End Generative Recommender that unifies item tokenization and generative recommendation into a cohesive framework.<n>ETEGRec consists of an item tokenizer and a generative recommender built on a dual encoder-decoder architecture.<n>We develop an alternating optimization technique to ensure stable and efficient end-to-end training of the entire framework.
  arXiv  Detail & Related papers  (2024-09-09T12:11:53Z)
• Analyzing and Enhancing the Backward-Pass Convergence of Unrolled Optimization [50.38518771642365]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which often lacks a closed form. This paper provides theoretical insights into the backward pass of unrolled optimization, showing that it is equivalent to the solution of a linear system by a particular iterative method. A system called Folded Optimization is proposed to construct more efficient backpropagation rules from unrolled solver implementations.
  arXiv  Detail & Related papers  (2023-12-28T23:15:18Z)
• Reconfigurable Distributed FPGA Cluster Design for Deep Learning Accelerators [59.11160990637615]
  We propose a distributed system based on lowpower embedded FPGAs designed for edge computing applications. The proposed system can simultaneously execute diverse Neural Network (NN) models, arrange the graph in a pipeline structure, and manually allocate greater resources to the most computationally intensive layers of the NN graph.
  arXiv  Detail & Related papers  (2023-05-24T16:08:55Z)
• MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation [8.070008246742681]
  State-of-the-art recommendation models rely on terabyte-scale embedding tables to learn user preferences. We show how synergies between embedding representations and hardware platforms can lead to improvements in both algorithmic- and system performance.
  arXiv  Detail & Related papers  (2023-02-21T18:38:45Z)
• Tailored Learning-Based Scheduling for Kubernetes-Oriented Edge-Cloud System [54.588242387136376]
  We introduce KaiS, a learning-based scheduling framework for edge-cloud systems. First, we design a coordinated multi-agent actor-critic algorithm to cater to decentralized request dispatch. Second, for diverse system scales and structures, we use graph neural networks to embed system state information. Third, we adopt a two-time-scale scheduling mechanism to harmonize request dispatch and service orchestration.
  arXiv  Detail & Related papers  (2021-01-17T03:45:25Z)
• MLComp: A Methodology for Machine Learning-based Performance Estimation and Adaptive Selection of Pareto-Optimal Compiler Optimization Sequences [10.200899224740871]
  We propose a novel Reinforcement Learning-based policy methodology for embedded software optimization. We show that different Machine Learning models are automatically tested to choose the best-fitting one. We also show that our framework can be trained efficiently for any target platform and application domain.
  arXiv  Detail & Related papers  (2020-12-09T19:13:39Z)
• Sapphire: Automatic Configuration Recommendation for Distributed Storage Systems [11.713288567936875]
  tuning parameters can provide significant performance gains but is a difficult task requiring profound experience and expertise. We propose an automatic simulation-based approach, Sapphire, to recommend optimal configurations. Results show that Sapphire significantly boosts Ceph performance to 2.2x compared to the default configuration.
  arXiv  Detail & Related papers  (2020-07-07T06:17:07Z)
• A Generic Network Compression Framework for Sequential Recommender Systems [71.81962915192022]
  Sequential recommender systems (SRS) have become the key technology in capturing user's dynamic interests and generating high-quality recommendations. We propose a compressed sequential recommendation framework, termed as CpRec, where two generic model shrinking techniques are employed. By the extensive ablation studies, we demonstrate that the proposed CpRec can achieve up to 4$sim$8 times compression rates in real-world SRS datasets.
  arXiv  Detail & Related papers  (2020-04-21T08:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.