Apache Submarine: A Unified Machine Learning Platform Made Simple
- URL: http://arxiv.org/abs/2108.09615v1
- Date: Sun, 22 Aug 2021 01:47:33 GMT
- Title: Apache Submarine: A Unified Machine Learning Platform Made Simple
- Authors: Kai-Hsun Chen, Huan-Ping Su, Wei-Chiu Chuang, Hung-Chang Hsiao, Wangda
Tan, Zhankun Tang, Xun Liu, Yanbo Liang, Wen-Chih Lo, Wanqiang Ji, Byron Hsu,
Keqiu Hu, HuiYang Jian, Quan Zhou, Chien-Min Wang
- Abstract summary: As machine learning is applied more widely, it is necessary to have a machine learning platform for both infrastructure administrators and users.
Existing machine learning platforms are ill-equipped to address the "Machine Learning tech debts" such as code, glue, and portability.
We propose Submarine, a unified machine learning platform, to address the challenges.
- Score: 3.667437879442565
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As machine learning is applied more widely, it is necessary to have a machine
learning platform for both infrastructure administrators and users including
expert data scientists and citizen data scientists to improve their
productivity. However, existing machine learning platforms are ill-equipped to
address the "Machine Learning tech debts" such as glue code, reproducibility,
and portability. Furthermore, existing platforms only take expert data
scientists into consideration, and thus they are inflexible for infrastructure
administrators and non-user-friendly for citizen data scientists. We propose
Submarine, a unified machine learning platform, to address the challenges.
Related papers
- A Declarative Query Language for Scientific Machine Learning [0.0]
We introduce a new declarative machine learning query language, called em MQL, for naive users.
We discuss two materials science experiments implemented using MQL on a materials science workflow system called MatFlow.
arXiv Detail & Related papers (2024-05-25T09:58:33Z) - GRID: A Platform for General Robot Intelligence Development [22.031523876249484]
We present a new platform for General Robot Intelligence Development (GRID)
The platform enables robots to learn, compose and adapt skills to their physical capabilities, environmental constraints and goals.
GRID is designed from the ground up to accommodate new types of robots, vehicles, hardware platforms and software protocols.
arXiv Detail & Related papers (2023-10-02T04:09:27Z) - GNM: A General Navigation Model to Drive Any Robot [67.40225397212717]
General goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots.
We analyze the necessary design decisions for effective data sharing across robots.
We deploy the trained GNM on a range of new robots, including an under quadrotor.
arXiv Detail & Related papers (2022-10-07T07:26:41Z) - A Survey of Machine Unlearning [56.017968863854186]
Recent regulations now require that, on request, private information about a user must be removed from computer systems.
ML models often remember' the old data.
Recent works on machine unlearning have not been able to completely solve the problem.
arXiv Detail & Related papers (2022-09-06T08:51:53Z) - Flashlight: Enabling Innovation in Tools for Machine Learning [50.63188263773778]
We introduce Flashlight, an open-source library built to spur innovation in machine learning tools and systems.
We see Flashlight as a tool enabling research that can benefit widely used libraries downstream and bring machine learning and systems researchers closer together.
arXiv Detail & Related papers (2022-01-29T01:03:29Z) - SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines.
This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
arXiv Detail & Related papers (2021-12-22T14:45:37Z) - Scientific Machine Learning Benchmarks [0.17205106391379021]
The breakthrough in Deep Learning neural networks has transformed the use of AI and machine learning technologies for the analysis of very large experimental datasets.
Identifying the most appropriate machine learning algorithm for the analysis of any given scientific dataset is still a challenge for scientists.
We describe our approach to the development of scientific machine learning benchmarks and review other approaches to benchmarking scientific machine learning.
arXiv Detail & Related papers (2021-10-25T10:05:11Z) - Ten Quick Tips for Deep Learning in Biology [116.78436313026478]
Machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling.
Deep learning has become its own subfield of machine learning.
In the context of biological research, deep learning has been increasingly used to derive novel insights from high-dimensional biological data.
arXiv Detail & Related papers (2021-05-29T21:02:44Z) - Synthetic Data: Opening the data floodgates to enable faster, more
directed development of machine learning methods [96.92041573661407]
Many ground-breaking advancements in machine learning can be attributed to the availability of a large volume of rich data.
Many large-scale datasets are highly sensitive, such as healthcare data, and are not widely available to the machine learning community.
Generating synthetic data with privacy guarantees provides one such solution.
arXiv Detail & Related papers (2020-12-08T17:26:10Z) - HAMLET: A Hierarchical Agent-based Machine Learning Platform [0.0]
HAMLET (Hierarchical Agent-based Machine LEarning plaTform) is a hybrid machine learning platform based on hierarchical multi-agent systems.
The proposed system models a machine learning solutions as a hypergraph and autonomously sets up a multi-level structure of heterogeneous agents.
arXiv Detail & Related papers (2020-10-10T03:46:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.