Modyn: Data-Centric Machine Learning Pipeline Orchestration

• URL: http://arxiv.org/abs/2312.06254v2
• Date: Mon, 25 Nov 2024 08:46:53 GMT
• Title: Modyn: Data-Centric Machine Learning Pipeline Orchestration
• Authors: Maximilian Böther, Ties Robroek, Viktor Gsteiger, Robin Holzinger, Xianzhe Ma, Pınar Tözün, Ana Klimovic,
• Abstract summary: Modyn is a data-centric end-to-end machine learning platform. We present Modyn, a data-centric end-to-end machine learning platform.
• Score: 1.4448995242976572
• License:
• Abstract: In real-world machine learning (ML) pipelines, datasets are continuously growing. Models must incorporate this new training data to improve generalization and adapt to potential distribution shifts. The cost of model retraining is proportional to how frequently the model is retrained and how much data it is trained on, which makes the naive approach of retraining from scratch each time impractical. We present Modyn, a data-centric end-to-end machine learning platform. Modyn's ML pipeline abstraction enables users to declaratively describe policies for continuously training a model on a growing dataset. Modyn pipelines allow users to apply data selection policies (to reduce the number of data points) and triggering policies (to reduce the number of trainings). Modyn executes and orchestrates these continuous ML training pipelines. The system is open-source and comes with an ecosystem of benchmark datasets, models, and tooling. We formally discuss how to measure the performance of ML pipelines by introducing the concept of composite models, enabling fair comparison of pipelines with different data selection and triggering policies. We empirically analyze how various data selection and triggering policies impact model accuracy, and also show that Modyn enables high throughput training with sample-level data selection.

Related papers

• Distilled Datamodel with Reverse Gradient Matching [74.75248610868685]
  We introduce an efficient framework for assessing data impact, comprising offline training and online evaluation stages. Our proposed method achieves comparable model behavior evaluation while significantly speeding up the process compared to the direct retraining method.
  arXiv  Detail & Related papers  (2024-04-22T09:16:14Z)
• No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance [68.18779562801762]
  multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance. Our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.
  arXiv  Detail & Related papers  (2024-04-04T17:58:02Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• Universal Metric Learning with Parameter-Efficient Transfer Learning [40.85295050164728]
  A common practice in metric learning is to train and test an embedding model for each dataset. This dataset-specific approach fails to simulate real-world scenarios that involve multiple heterogeneous distributions of data. We introduce a novel metric learning paradigm, called Universal Metric Learning (UML), which learns a unified metric capable of capturing relations across multiple data distributions.
  arXiv  Detail & Related papers  (2023-09-16T10:34:01Z)
• SOTASTREAM: A Streaming Approach to Machine Translation Training [13.39347756245191]
  Many machine translation toolkits make use of a data preparation step wherein raw data is transformed into a tensor format that can be used directly by the trainer. We propose an alternative approach that separates the generation of data from the consumption of that data. In this approach, there is no separate pre-processing step; data generation produces an infinite stream of permutations of the raw training data.
  arXiv  Detail & Related papers  (2023-08-14T22:47:19Z)
• TRAK: Attributing Model Behavior at Scale [79.56020040993947]
  We present TRAK (Tracing with Randomly-trained After Kernel), a data attribution method that is both effective and computationally tractable for large-scale, differenti models.
  arXiv  Detail & Related papers  (2023-03-24T17:56:22Z)
• Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning [119.70303730341938]
  We propose ePisode cUrriculum inveRsion (ECI) during data-free meta training and invErsion calibRation following inner loop (ICFIL) during meta testing. ECI adaptively increases the difficulty level of pseudo episodes according to the real-time feedback of the meta model. We formulate the optimization process of meta training with ECI as an adversarial form in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-03-20T15:10:41Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines [27.461398584509755]
  DataScope is the first system that efficiently computes Shapley values of training examples over an end-to-end machine learning pipeline. Our results show that DataScope is up to four orders of magnitude faster than state-of-the-art Monte Carlo-based methods.
  arXiv  Detail & Related papers  (2022-04-23T19:29:23Z)
• It's the Best Only When It Fits You Most: Finding Related Models for Serving Based on Dynamic Locality Sensitive Hashing [1.581913948762905]
  Preparation of training data is often a bottleneck in the lifecycle of deploying a deep learning model for production or research. This paper proposes an end-to-end process of searching related models for serving based on the similarity of the target dataset and the training datasets of the available models.
  arXiv  Detail & Related papers  (2020-10-13T22:52:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.