DiffML: End-to-end Differentiable ML Pipelines

• URL: http://arxiv.org/abs/2207.01269v2
• Date: Tue, 5 Jul 2022 07:39:32 GMT
• Title: DiffML: End-to-end Differentiable ML Pipelines
• Authors: Benjamin Hilprecht, Christian Hammacher, Eduardo Reis, Mohamed Abdelaal and Carsten Binnig
• Abstract summary: DiffML allows to jointly train not just the ML model itself but also the entire pipeline. Our core idea is to formulate all pipeline steps in a differentiable way. This is a non-trivial problem and opens up many new research questions.
• Score: 12.869023436690894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present our vision of differentiable ML pipelines called DiffML to automate the construction of ML pipelines in an end-to-end fashion. The idea is that DiffML allows to jointly train not just the ML model itself but also the entire pipeline including data preprocessing steps, e.g., data cleaning, feature selection, etc. Our core idea is to formulate all pipeline steps in a differentiable way such that the entire pipeline can be trained using backpropagation. However, this is a non-trivial problem and opens up many new research questions. To show the feasibility of this direction, we demonstrate initial ideas and a general principle of how typical preprocessing steps such as data cleaning, feature selection and dataset selection can be formulated as differentiable programs and jointly learned with the ML model. Moreover, we discuss a research roadmap and core challenges that have to be systematically tackled to enable fully differentiable ML pipelines.

Related papers

• Instrumentation and Analysis of Native ML Pipelines via Logical Query Plans [3.2362171533623054]
  We envision highly-automated software platforms to assist data scientists with developing, validating, monitoring, and analysing their Machine Learning pipelines. We extract "logical query plans" from ML pipeline code relying on popular libraries. Based on these plans, we automatically infer pipeline semantics and instrument and rewrite the ML pipelines to enable diverse use cases without requiring data scientists to manually annotate or rewrite their code.
  arXiv  Detail & Related papers  (2024-07-10T11:35:02Z)
• Optimal Flow Matching: Learning Straight Trajectories in Just One Step [89.37027530300617]
  We develop and theoretically justify the novel Optimal Flow Matching approach. It allows recovering the straight OT displacement for the quadratic transport in just one FM step. The main idea of our approach is the employment of vector field for FM which are parameterized by convex functions.
  arXiv  Detail & Related papers  (2024-03-19T19:44:54Z)
• Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization [87.21285093582446]
  Diffusion Generative Flow Samplers (DGFS) is a sampling-based framework where the learning process can be tractably broken down into short partial trajectory segments. Our method takes inspiration from the theory developed for generative flow networks (GFlowNets)
  arXiv  Detail & Related papers  (2023-10-04T09:39:05Z)
• REIN: A Comprehensive Benchmark Framework for Data Cleaning Methods in ML Pipelines [0.0]
  We introduce a benchmark, called REIN1, to thoroughly investigate the impact of data cleaning methods on various machine learning models. Through the benchmark, we provide answers to important research questions, e.g., where and whether data cleaning is a necessary step in ML pipelines.
  arXiv  Detail & Related papers  (2023-02-09T15:37:39Z)
• Modeling Quality and Machine Learning Pipelines through Extended Feature Models [0.0]
  We propose a new engineering approach for quality ML pipeline by properly extending the Feature Models meta-model. The presented approach allows to model ML pipelines, their quality requirements (on the whole pipeline and on single phases) and quality characteristics of algorithms used to implement each pipeline phase.
  arXiv  Detail & Related papers  (2022-07-15T15:20:28Z)
• Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines [27.461398584509755]
  DataScope is the first system that efficiently computes Shapley values of training examples over an end-to-end machine learning pipeline. Our results show that DataScope is up to four orders of magnitude faster than state-of-the-art Monte Carlo-based methods.
  arXiv  Detail & Related papers  (2022-04-23T19:29:23Z)
• Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference [74.80730361332711]
  Few-shot learning is an important and topical problem in computer vision. We show that a simple transformer-based pipeline yields surprisingly good performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-15T02:55:58Z)
• Adaptive neighborhood Metric learning [184.95321334661898]
  We propose a novel distance metric learning algorithm, named adaptive neighborhood metric learning (ANML) ANML can be used to learn both the linear and deep embeddings. The emphlog-exp mean function proposed in our method gives a new perspective to review the deep metric learning methods.
  arXiv  Detail & Related papers  (2022-01-20T17:26:37Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• A Rigorous Machine Learning Analysis Pipeline for Biomedical Binary Classification: Application in Pancreatic Cancer Nested Case-control Studies with Implications for Bias Assessments [2.9726886415710276]
  We have laid out and assembled a complete, rigorous ML analysis pipeline focused on binary classification. This 'automated' but customizable pipeline includes a) exploratory analysis, b) data cleaning and transformation, c) feature selection, d) model training with 9 established ML algorithms. We apply this pipeline to an epidemiological investigation of established and newly identified risk factors for cancer to evaluate how different sources of bias might be handled by ML algorithms.
  arXiv  Detail & Related papers  (2020-08-28T19:58:05Z)
• Semi-Supervised Learning with Normalizing Flows [54.376602201489995]
  FlowGMM is an end-to-end approach to generative semi supervised learning with normalizing flows. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data.
  arXiv  Detail & Related papers  (2019-12-30T17:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.