SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices

• URL: http://arxiv.org/abs/2109.03947v1
• Date: Wed, 8 Sep 2021 22:06:16 GMT
• Title: SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices
• Authors: Chulhong Min, Akhil Mathur, Utku Gunay Acer, Alessandro Montanari, Fahim Kawsar
• Abstract summary: We present a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. S SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.
• Score: 69.1412199244903
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present SensiX++ - a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. First, a data coordinator manages the lifecycle of sensors and serves models with correct data through automated transformations. Next, a resource-aware model server executes multiple models in isolation through model abstraction, pipeline automation and feature sharing. An adaptive scheduler then orchestrates the best-effort executions of multiple models across heterogeneous accelerators, balancing latency and throughput. Finally, microservices with REST APIs serve synthesised model predictions, system statistics, and continuous deployment. Collectively, these components enable SensiX++ to serve multiple models efficiently with fine-grained control on edge devices while minimising data operation redundancy, managing data and device heterogeneity, reducing resource contention and removing manual MLOps. We benchmark SensiX++ with ten different vision and acoustics models across various multi-tenant configurations on different edge accelerators (Jetson AGX and Coral TPU) designed for sensory devices. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.

Related papers

• STAMP: Scalable Task And Model-agnostic Collaborative Perception [24.890993164334766]
  STAMP is a task- and model-agnostic, collaborative perception pipeline for heterogeneous agents. It minimizes computational overhead, enhances scalability, and preserves model security. As a first-of-its-kind framework, STAMP aims to advance research in scalable and secure mobility systems towards Level 5 autonomy.
  arXiv  Detail & Related papers  (2025-01-24T16:27:28Z)
• MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference [4.556037016746581]
  We introduce MultiTASC++, a continuously adaptive multi-tenancy-aware scheduler for distributed inference. We demonstrate the scheduler's efficacy in consistently maintaining a targeted satisfaction rate while providing the highest available accuracy across different device tiers and workloads of up to 100 devices.
  arXiv  Detail & Related papers  (2024-12-05T13:19:34Z)
• OminiControl: Minimal and Universal Control for Diffusion Transformer [68.3243031301164]
  OminiControl is a framework that integrates image conditions into pre-trained Diffusion Transformer (DiT) models. At its core, OminiControl leverages a parameter reuse mechanism, enabling the DiT to encode image conditions using itself as a powerful backbone. OminiControl addresses a wide range of image conditioning tasks in a unified manner, including subject-driven generation and spatially-aligned conditions.
  arXiv  Detail & Related papers  (2024-11-22T17:55:15Z)
• From One to the Power of Many: Invariance to Multi-LiDAR Perception from Single-Sensor Datasets [12.712896458348515]
  We introduce a new metric for feature-level invariance which can serve as a proxy to measure cross-domain generalization without requiring labeled data. We propose two application-specific data augmentations, which facilitate better transfer to multi-sensor setups LiDAR, when trained on single-sensor datasets.
  arXiv  Detail & Related papers  (2024-09-27T09:51:45Z)
• Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration [37.456185990843515]
  We introduce a Universal On-Device Multi-modal Model Adaptation Framework. The framework features the Fast Domain Adaptor (FDA) hosted in the cloud, providing tailored parameters for the Lightweight Multi-modal Model on devices. Our contributions represent a pioneering solution for on-Device Multi-modal Model Adaptation (DMMA)
  arXiv  Detail & Related papers  (2024-05-21T14:42:18Z)
• A Generative Approach for Production-Aware Industrial Network Traffic Modeling [70.46446906513677]
  We investigate the network traffic data generated from a laser cutting machine deployed in a Trumpf factory in Germany. We analyze the traffic statistics, capture the dependencies between the internal states of the machine, and model the network traffic as a production state dependent process. We compare the performance of various generative models including variational autoencoder (VAE), conditional variational autoencoder (CVAE), and generative adversarial network (GAN)
  arXiv  Detail & Related papers  (2022-11-11T09:46:58Z)
• DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization [66.27399823422665]
  Device Model Generalization (DMG) is a practical yet under-investigated research topic for on-device machine learning applications. We propose an efficient Device-cloUd collaborative parametErs generaTion framework DUET.
  arXiv  Detail & Related papers  (2022-09-12T13:26:26Z)
• Edge Federated Learning Via Unit-Modulus Over-The-Air Computation (Extended Version) [64.76619508293966]
  This paper proposes a unit-modulus over-the-air computation (UM-AirComp) framework to facilitate efficient edge federated learning. It uploads simultaneously local model parameters and updates global model parameters via analog beamforming. We demonstrate the implementation of UM-AirComp in a vehicle-to-everything autonomous driving simulation platform.
  arXiv  Detail & Related papers  (2021-01-28T15:10:22Z)
• SensiX: A Platform for Collaborative Machine Learning on the Edge [69.1412199244903]
  We present SensiX, a personal edge platform that stays between sensor data and sensing models. We demonstrate its efficacy in developing motion and audio-based multi-device sensing systems. Our evaluation shows that SensiX offers a 7-13% increase in overall accuracy and up to 30% increase across different environment dynamics at the expense of 3mW power overhead.
  arXiv  Detail & Related papers  (2020-12-04T23:06:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.