YMIR: A Rapid Data-centric Development Platform for Vision Applications
- URL: http://arxiv.org/abs/2111.10046v1
- Date: Fri, 19 Nov 2021 05:02:55 GMT
- Title: YMIR: A Rapid Data-centric Development Platform for Vision Applications
- Authors: Phoenix X. Huang, Wenze Hu, William Brendel, Manmohan Chandraker,
Li-Jia Li, Xiaoyu Wang
- Abstract summary: This paper introduces an open source platform for rapid development of computer vision applications.
The platform puts the efficient data development at the center of the machine learning development process.
- Score: 82.67319997259622
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper introduces an open source platform for rapid development of
computer vision applications. The platform puts the efficient data development
at the center of the machine learning development process, integrates active
learning methods, data and model version control, and uses concepts such as
projects to enable fast iteration of multiple task specific datasets in
parallel. We make it an open platform by abstracting the development process
into core states and operations, and design open APIs to integrate third party
tools as implementations of the operations. This open design reduces the
development cost and adoption cost for ML teams with existing tools. At the
same time, the platform supports recording project development history, through
which successful projects can be shared to further boost model production
efficiency on similar tasks. The platform is open source and is already used
internally to meet the increasing demand from custom real world computer vision
applications.
Related papers
- OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models [61.14336781917986]
We introduce OpenR, an open-source framework for enhancing the reasoning capabilities of large language models (LLMs)
OpenR unifies data acquisition, reinforcement learning training, and non-autoregressive decoding into a cohesive software platform.
Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning.
arXiv Detail & Related papers (2024-10-12T23:42:16Z) - Automatic Platform Configuration and Software Integration for Software-Defined Vehicles [4.522485108591059]
This paper introduces a novel approach to automate platform configuration and software integration for software-defined vehicles (SDVs)
By leveraging model-based systems engineering (MBSE), our method automatically generates platform configuration and software integration plans.
The proposed system enables dynamic and flexible resource allocation while ensuring compliance with safety requirements.
arXiv Detail & Related papers (2024-08-04T19:54:03Z) - ExaWorks Software Development Kit: A Robust and Scalable Collection of Interoperable Workflow Technologies [3.1805622006446397]
Heterogeneous scientific discovery increasingly requires executing on high-performance computing platforms.
We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK)
The SDK is a collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms.
arXiv Detail & Related papers (2024-07-23T17:00:09Z) - Emerging Platforms Meet Emerging LLMs: A Year-Long Journey of Top-Down Development [20.873143073842705]
We introduce TapML, a top-down approach and tooling designed to streamline the deployment of machine learning systems on diverse platforms.
Unlike traditional bottom-up methods, TapML automates unit testing and adopts a migration-based strategy for gradually offloading model computations.
TapML was developed and applied through a year-long, real-world effort that successfully deployed significant emerging models and platforms.
arXiv Detail & Related papers (2024-04-14T06:09:35Z) - The GitHub Development Workflow Automation Ecosystems [47.818229204130596]
Large-scale software development has become a highly collaborative endeavour.
This chapter explores the ecosystems of development bots and GitHub Actions.
It provides an extensive survey of the state-of-the-art in this domain.
arXiv Detail & Related papers (2023-05-08T15:24:23Z) - A Scalable Approach to Modeling on Accelerated Neuromorphic Hardware [0.0]
This work presents the software aspects of the BrainScaleS-2 system, a hybrid accelerated neuromorphic hardware architecture based on physical modeling.
We introduce key aspects of the BrainScaleS-2 Operating System: experiment workflow, API layering, software design, and platform operation.
The focus lies on novel system and software features such as multi-compartmental neurons, fast re-configuration for hardware-in-the-loop training, applications for the embedded processors, the non-spiking operation mode, interactive platform access, and sustainable hardware/software co-development.
arXiv Detail & Related papers (2022-03-21T16:30:18Z) - Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data
Programming [77.38174112525168]
We present Nemo, an end-to-end interactive Supervision system that improves overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS supervision approach.
arXiv Detail & Related papers (2022-03-02T19:57:32Z) - SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines.
This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
arXiv Detail & Related papers (2021-12-22T14:45:37Z) - Knowledge Integration of Collaborative Product Design Using Cloud
Computing Infrastructure [65.2157099438235]
The main focus of this paper is the concept of ongoing research in providing the knowledge integration service for collaborative product design and development using cloud computing infrastructure.
Proposed knowledge integration services support users by giving real-time access to knowledge resources.
arXiv Detail & Related papers (2020-01-16T18:44:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.