Quota Marketplace: Dynamic Pricing for Efficient Allocation of ML Training Resources

• URL: http://arxiv.org/abs/2607.09802v1
• Date: Thu, 09 Jul 2026 17:24:35 GMT
• Title: Quota Marketplace: Dynamic Pricing for Efficient Allocation of ML Training Resources
• Abstract summary: This paper describes the design, implementation, deployment, and theoretical analysis of Quota Marketplace.<n>It is a market-based mechanism to efficiently allocate Machine Learning (ML) training chips.<n>We discuss many business-critical requirements that the Quota Marketplace handles quite effectively.
• Score: 27.710446752525076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The escalating demand for Machine Learning (ML) training resources in recent years has resulted in a substantial gap between the high demand and the available supply. Efficient allocation of these scarce and expensive resources is crucial for organizations to maximize their return on investment. Existing resource allocation mechanisms, like Karma [OSDI'23], are designed to guarantee Pareto efficiency and max-min fairness in settings with dynamic (time-varying) user demands, but fail to preserve these key properties in the presence of demands with heterogeneous values. Given the ubiquity and inevitability of heterogeneity in organizational values of different workloads, effective resource allocation policies must accommodate these variations. In this paper, we describe the design, implementation, deployment, and theoretical analysis of Quota Marketplace, a market-based mechanism to efficiently allocate ML training chips (like GPUs), explicitly addressing scenarios with demands of heterogeneous value. We detail the implementation of this mechanism within Google and present metrics that demonstrate its impact. We also discuss many business-critical requirements that the Quota Marketplace handles quite effectively, and document the gains and opportunities it has unlocked. We establish theoretically how this market-based approach achieves the essential properties of Pareto efficiency and max-min fairness by allowing the users to express the value of their workloads and enabling dynamic resource pricing based on supply and demand fluctuations. Ultimately, the market facilitates resource allocation that aligns with organizational priorities.

Related papers

• ScaleSense: Cost-Intelligent Scaling Framework via Learned Resource Estimation in Alibaba AnalyticDB [59.708423132520515]
  Cloud-native serverless data warehouses achieve fine-grained elasticity by decoupling storage from compute.<n>Yet determining the optimal resource allocation for highly heterogeneous ad-hoc queries remains a formidable industrial challenge.<n>We propose ScaleSense, a proactive, query-level resource scaling framework.
  arXiv  Detail & Related papers  (2026-08-08T06:10:35Z)
• Trading Utility for Dynamic Fairness in Multiple Resource Division with Sequential Demand [8.838518847441728]
  We propose a neural allocation mechanism that reconciles fairness with utility.<n>We first formalize fairness in the dynamic setting via stepwise loss functions.<n>We parameterized the solutions by constraining allocations to the subspace of demand.
  arXiv  Detail & Related papers  (2026-06-09T06:41:04Z)
• Bayesian Optimization for Dynamic Pricing and Learning [0.306238659426286]
  Dynamic pricing is the practice of adjusting the selling price of a product to maximize a firm's revenue by responding to market demand.<n>Traditional approaches often assume a specific parametric form for the demand function, enabling the use of reinforcement learning (RL) to identify near-optimal pricing strategies.<n>We propose a nonparametric approach to dynamic pricing that avoids restrictive modeling assumptions.
  arXiv  Detail & Related papers  (2025-10-14T12:28:06Z)
• Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading [57.28635022507172]
  TiMi is a rationality-driven multi-agent system that architecturally decouples strategy development from minute-level deployment.<n>We propose a two-tier analytical paradigm from macro patterns to micro customization, layered programming design for trading bot implementation, and closed-loop optimization driven by mathematical reflection.
  arXiv  Detail & Related papers  (2025-10-06T13:08:55Z)
• MOHAF: A Multi-Objective Hierarchical Auction Framework for Scalable and Fair Resource Allocation in IoT Ecosystems [0.565395466029518]
  This paper proposes a distributed resource allocation mechanism that jointly optimize cost, Quality of Service (QoS), energy efficiency, and fairness.<n>Experiments on the Google Cluster Data trace, comprising 3,553 requests and 888 resources, demonstrate MOHAF's superior allocation efficiency (0.263 compared to Greedy (0.185), First-Price (0.138), and Random (0.101) auctions, while achieving perfect fairness (Jain's index = 1.000).
  arXiv  Detail & Related papers  (2025-08-20T16:25:37Z)
• COALESCE: Economic and Security Dynamics of Skill-Based Task Outsourcing Among Team of Autonomous LLM Agents [0.0]
  COALESCE is a novel framework designed to enable autonomous Large Language Model (LLM) agents to outsource specific subtasks to specialized, cost-effective third-party LLM agents.<n> Comprehensive validation through 239 theoretical simulations demonstrates 41.8% cost reduction potential.<n>Large-scale empirical validation across 240 real LLM tasks confirms 20.3% cost reduction with proper epsilon-greedy exploration.
  arXiv  Detail & Related papers  (2025-06-02T17:22:47Z)
• Deep Learning Meets Mechanism Design: Key Results and Some Novel Applications [1.2661010067882734]
  We present, from relevant literature, technical details of using a deep learning approach for mechanism design. We demonstrate the power of this approach for three illustrative case studies.
  arXiv  Detail & Related papers  (2024-01-11T06:09:32Z)
• HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE [113.47287249524008]
  It is still an open question to build a factor model that can conduct stock prediction in an online and adaptive setting. We propose the first deep learning based online and adaptive factor model, HireVAE, at the core of which is a hierarchical latent space that embeds the relationship between the market situation and stock-wise latent factors. Across four commonly used real stock market benchmarks, the proposed HireVAE demonstrate superior performance in terms of active returns over previous methods.
  arXiv  Detail & Related papers  (2023-06-05T12:58:13Z)
• Online Learning of Competitive Equilibria in Exchange Economies [94.24357018178867]
  In economics, the sharing of scarce resources among multiple rational agents is a classical problem. We propose an online learning mechanism to learn agent preferences. We demonstrate the effectiveness of this mechanism through numerical simulations.
  arXiv  Detail & Related papers  (2021-06-11T21:32:17Z)
• Online Learning Demands in Max-min Fairness [91.37280766977923]
  We describe mechanisms for the allocation of a scarce resource among multiple users in a way that is efficient, fair, and strategy-proof. The mechanism is repeated for multiple rounds and a user's requirements can change on each round. At the end of each round, users provide feedback about the allocation they received, enabling the mechanism to learn user preferences over time.
  arXiv  Detail & Related papers  (2020-12-15T22:15:20Z)
• VCG Mechanism Design with Unknown Agent Values under Stochastic Bandit Feedback [104.06766271716774]
  We study a multi-round welfare-maximising mechanism design problem in instances where agents do not know their values. We first define three notions of regret for the welfare, the individual utilities of each agent and that of the mechanism. Our framework also provides flexibility to control the pricing scheme so as to trade-off between the agent and seller regrets.
  arXiv  Detail & Related papers  (2020-04-19T18:00:58Z)
• Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning [100.73223416589596]
  We propose a cost-sensitive portfolio selection method with deep reinforcement learning. Specifically, a novel two-stream portfolio policy network is devised to extract both price series patterns and asset correlations. A new cost-sensitive reward function is developed to maximize the accumulated return and constrain both costs via reinforcement learning.
  arXiv  Detail & Related papers  (2020-03-06T06:28:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.