Scaling ML Products At Startups: A Practitioner's Guide

• URL: http://arxiv.org/abs/2304.10660v1
• Date: Thu, 20 Apr 2023 22:02:42 GMT
• Title: Scaling ML Products At Startups: A Practitioner's Guide
• Authors: Atul Dhingra, Gaurav Sood
• Abstract summary: We break down costs into variable costs-the cost of serving the model and performant-and fixed costs-the cost of developing and training new models. We propose a framework for conceptualizing these costs, breaking them into finer categories, and limn ways to reduce costs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How do you scale a machine learning product at a startup? In particular, how do you serve a greater volume, velocity, and variety of queries cost-effectively? We break down costs into variable costs-the cost of serving the model and performant-and fixed costs-the cost of developing and training new models. We propose a framework for conceptualizing these costs, breaking them into finer categories, and limn ways to reduce costs. Lastly, since in our experience, the most expensive fixed cost of a machine learning system is the cost of identifying the root causes of failures and driving continuous improvement, we present a way to conceptualize the issues and share our methodology for the same.

Related papers

• A Primal-Dual Online Learning Approach for Dynamic Pricing of Sequentially Displayed Complementary Items under Sale Constraints [54.46126953873298]
  We address the problem of dynamically pricing complementary items that are sequentially displayed to customers. Coherent pricing policies for complementary items are essential because optimizing the pricing of each item individually is ineffective. We empirically evaluate our approach using synthetic settings randomly generated from real-world data, and compare its performance in terms of constraints violation and regret.
  arXiv  Detail & Related papers  (2024-07-08T09:55:31Z)
• Power Hungry Processing: Watts Driving the Cost of AI Deployment? [74.19749699665216]
  generative, multi-purpose AI systems promise a unified approach to building machine learning (ML) models into technology. This ambition of generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions
  arXiv  Detail & Related papers  (2023-11-28T15:09:36Z)
• Cost-Effective Retraining of Machine Learning Models [2.9461360639852914]
  It is important to retrain a machine learning (ML) model in order to maintain its performance as the data changes over time. This creates a trade-off between retraining too frequently, which leads to unnecessary computing costs, and not retraining often enough. We propose ML systems that make automated and cost-effective decisions about when to retrain an ML model.
  arXiv  Detail & Related papers  (2023-10-06T13:02:29Z)
• A Penny a Function: Towards Cost Transparent Cloud Programming [3.858859576352153]
  Existing tools for understanding cost factors are often detached from source code. Existing cost models for cloud applications focus on specific factors such as compute resources. This paper presents initial work toward a cost model based on a directed graph that allows deriving monetary cost estimations directly from code.
  arXiv  Detail & Related papers  (2023-09-10T08:02:12Z)
• Accelerated and Inexpensive Machine Learning for Manufacturing Processes with Incomplete Mechanistic Knowledge [0.0]
  This paper proposes a transfer learning based approach to address this issue. A ML model is trained on a large amount of computationally inexpensive data from a physics-based process model (source) and then fine-tuned on a smaller amount of costly experimental data (target) Despite extreme functional and quantitative inaccuracies in the source our approach reduces the model development cost by years, experimental cost by 56-76%, computational cost by orders of magnitude, and prediction error by 16-24%.
  arXiv  Detail & Related papers  (2023-04-29T10:54:57Z)
• Optimizing Data Collection for Machine Learning [87.37252958806856]
  Modern deep learning systems require huge data sets to achieve impressive performance. Over-collecting data incurs unnecessary present costs, while under-collecting may incur future costs and delay. We propose a new paradigm for modeling the data collection as a formal optimal data collection problem.
  arXiv  Detail & Related papers  (2022-10-03T21:19:05Z)
• Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction [18.46293613612346]
  We introduce zero-shot cost models which enable learned cost estimation that generalizes to unseen databases. We suggest a new learning paradigm based on pre-trained cost models. We show that zero-shot cost models can be used in a few-shot mode that further improves their quality by retraining them just with a small number of additional training queries on the unseen database.
  arXiv  Detail & Related papers  (2022-01-03T10:11:35Z)
• Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions [74.00030431081751]
  We formalize the notion of user-specific cost functions and introduce a new method for identifying actionable recourses for users. Our method satisfies up to 25.89 percentage points more users compared to strong baseline methods.
  arXiv  Detail & Related papers  (2021-11-01T19:49:35Z)
• ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training [65.68511423300812]
  We propose ProgFed, a progressive training framework for efficient and effective federated learning. ProgFed inherently reduces computation and two-way communication costs while maintaining the strong performance of the final models. Our results show that ProgFed converges at the same rate as standard training on full models.
  arXiv  Detail & Related papers  (2021-10-11T14:45:00Z)
• Training Over-parameterized Models with Non-decomposable Objectives [46.62273918807789]
  We propose new cost-sensitive losses that extend the classical idea of logit adjustment to handle more general cost matrices. Our losses are calibrated, and can be further improved with distilled labels from a teacher model.
  arXiv  Detail & Related papers  (2021-07-09T19:29:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.