YMIR: A Rapid Data-centric Development Platform for Vision Applications
- URL: http://arxiv.org/abs/2111.10046v1
- Date: Fri, 19 Nov 2021 05:02:55 GMT
- Title: YMIR: A Rapid Data-centric Development Platform for Vision Applications
- Authors: Phoenix X. Huang, Wenze Hu, William Brendel, Manmohan Chandraker,
Li-Jia Li, Xiaoyu Wang
- Abstract summary: This paper introduces an open source platform for rapid development of computer vision applications.
The platform puts the efficient data development at the center of the machine learning development process.
- Score: 82.67319997259622
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper introduces an open source platform for rapid development of
computer vision applications. The platform puts the efficient data development
at the center of the machine learning development process, integrates active
learning methods, data and model version control, and uses concepts such as
projects to enable fast iteration of multiple task specific datasets in
parallel. We make it an open platform by abstracting the development process
into core states and operations, and design open APIs to integrate third party
tools as implementations of the operations. This open design reduces the
development cost and adoption cost for ML teams with existing tools. At the
same time, the platform supports recording project development history, through
which successful projects can be shared to further boost model production
efficiency on similar tasks. The platform is open source and is already used
internally to meet the increasing demand from custom real world computer vision
applications.
Related papers
- ExaWorks Software Development Kit: A Robust and Scalable Collection of Interoperable Workflow Technologies [3.1805622006446397]
Heterogeneous scientific discovery increasingly requires executing on high-performance computing platforms.
We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK)
The SDK is a collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms.
arXiv Detail & Related papers (2024-07-23T17:00:09Z) - Emerging Platforms Meet Emerging LLMs: A Year-Long Journey of Top-Down Development [20.873143073842705]
We introduce TapML, a top-down approach and tooling designed to streamline the deployment of machine learning systems on diverse platforms.
Unlike traditional bottom-up methods, TapML automates unit testing and adopts a migration-based strategy for gradually offloading model computations.
TapML was developed and applied through a year-long, real-world effort that successfully deployed significant emerging models and platforms.
arXiv Detail & Related papers (2024-04-14T06:09:35Z) - The GitHub Development Workflow Automation Ecosystems [47.818229204130596]
Large-scale software development has become a highly collaborative endeavour.
This chapter explores the ecosystems of development bots and GitHub Actions.
It provides an extensive survey of the state-of-the-art in this domain.
arXiv Detail & Related papers (2023-05-08T15:24:23Z) - On Efficient Training of Large-Scale Deep Learning Models: A Literature
Review [90.87691246153612]
The field of deep learning has witnessed significant progress, particularly in computer vision (CV), natural language processing (NLP), and speech.
The use of large-scale models trained on vast amounts of data holds immense promise for practical applications.
With the increasing demands on computational capacity, a comprehensive summarization on acceleration techniques of training deep learning models is still much anticipated.
arXiv Detail & Related papers (2023-04-07T11:13:23Z) - A Scalable Approach to Modeling on Accelerated Neuromorphic Hardware [0.0]
This work presents the software aspects of the BrainScaleS-2 system, a hybrid accelerated neuromorphic hardware architecture based on physical modeling.
We introduce key aspects of the BrainScaleS-2 Operating System: experiment workflow, API layering, software design, and platform operation.
The focus lies on novel system and software features such as multi-compartmental neurons, fast re-configuration for hardware-in-the-loop training, applications for the embedded processors, the non-spiking operation mode, interactive platform access, and sustainable hardware/software co-development.
arXiv Detail & Related papers (2022-03-21T16:30:18Z) - Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data
Programming [77.38174112525168]
We present Nemo, an end-to-end interactive Supervision system that improves overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS supervision approach.
arXiv Detail & Related papers (2022-03-02T19:57:32Z) - SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines.
This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
arXiv Detail & Related papers (2021-12-22T14:45:37Z) - A Pipeline for Vision-Based On-Orbit Proximity Operations Using Deep
Learning and Synthetic Imagery [0.0]
Two key challenges currently pose a major barrier to the use of deep learning for vision-based on-orbit proximity operations.
A scarcity of labeled training data (images of a target spacecraft) hinders creation of robust deep learning models.
This paper presents an open-source deep learning pipeline, developed specifically for on-orbit visual navigation applications.
arXiv Detail & Related papers (2021-01-14T15:17:54Z) - Knowledge Integration of Collaborative Product Design Using Cloud
Computing Infrastructure [65.2157099438235]
The main focus of this paper is the concept of ongoing research in providing the knowledge integration service for collaborative product design and development using cloud computing infrastructure.
Proposed knowledge integration services support users by giving real-time access to knowledge resources.
arXiv Detail & Related papers (2020-01-16T18:44:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.