An Ultra-low Power TinyML System for Real-time Visual Processing at Edge

• URL: http://arxiv.org/abs/2207.04663v2
• Date: Thu, 1 Jun 2023 06:49:38 GMT
• Title: An Ultra-low Power TinyML System for Real-time Visual Processing at Edge
• Abstract summary: This brief presents an extremely tiny backbone to construct high efficiency CNN models for various visual tasks. A specially designed neural co-processor (NCP) is interconnected with MCU to build an ultra-low power TinyML system. The proposed TinyML system based on our model, NCP and instruction set yields considerable accuracy and achieves a record ultra-low power of 160mW.
• Score: 7.327401565768275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tiny machine learning (TinyML), executing AI workloads on resource and power strictly restricted systems, is an important and challenging topic. This brief firstly presents an extremely tiny backbone to construct high efficiency CNN models for various visual tasks. Then, a specially designed neural co-processor (NCP) is interconnected with MCU to build an ultra-low power TinyML system, which stores all features and weights on chip and completely removes both of latency and power consumption in off-chip memory access. Furthermore, an application specific instruction-set is further presented for realizing agile development and rapid deployment. Extensive experiments demonstrate that the proposed TinyML system based on our model, NCP and instruction set yields considerable accuracy and achieves a record ultra-low power of 160mW while implementing object detection and recognition at 30FPS. The demo video is available on \url{https://www.youtube.com/watch?v=mIZPxtJ-9EY}.

Related papers

• Power-Performance Characterization of TinyML Systems [23.93944681214181]
  This paper presents a systematic performance and power characterization of diverse TinyML applications on microcontrollers (MCUs)<n>We focus on the impact of the multiple layers of abstraction that provide higher programmability at the expense of performance and energy efficiency.
  arXiv  Detail & Related papers  (2026-08-21T21:34:24Z)
• NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics [50.594459728605734]
  Small form factor, i.e., a few 10s grams, severely limits onboard computational resources to sub-SI100milliwatt microcontroller units (MCUs)<n>Our framework achieves ideal end-to-end latency, i.e. zero overhead due to serialized tasks, delivering quantifiable improvements in closed-loop control performance.
  arXiv  Detail & Related papers  (2026-01-12T12:29:38Z)
• RockNet: Distributed Learning on Ultra-Low-Power Devices [49.01692357536576]
  This paper presents RockNet, a new TinyML method tailored for ultra-low-power hardware.<n>By leveraging that CPS consist of multiple devices, we design a distributed learning method that integrates Machine Learning and wireless communication.<n>Our results show that a tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2025-10-15T09:09:30Z)
• Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training [16.821900475733102]
  Tin-Tin is an integer-based on-device training framework for low-power microcontrollers (MCUs) We introduce novel integer rescaling techniques to efficiently manage dynamic ranges and facilitate efficient weight updates. We validate the effectiveness of Tin-Tin through end-to-end application examples on real-world tiny devices.
  arXiv  Detail & Related papers  (2025-04-13T02:21:24Z)
• LSNet: See Large, Focus Small [67.05569159984691]
  We introduce LS (textbfLarge-textbfSmall) convolution, which combines large- kernel perception and small- kernel aggregation. LSNet achieves superior performance and efficiency over existing lightweight networks in various vision tasks.
  arXiv  Detail & Related papers  (2025-03-29T16:00:54Z)
• Fast Data Aware Neural Architecture Search via Supernet Accelerated Evaluation [0.43550340493919387]
  Tiny machine learning (TinyML) promises to revolutionize fields such as healthcare, environmental monitoring, and industrial maintenance. The complex optimizations required for successful TinyML deployment continue to impede its widespread adoption. We propose a new state-of-the-art Data Aware Neural Architecture Search technique and demonstrate its effectiveness on the novel TinyML VisionWake' dataset.
  arXiv  Detail & Related papers  (2025-02-18T09:51:03Z)
• DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution [114.61347672265076]
  Development of MLLMs for real-world robots is challenging due to the typically limited computation and memory capacities available on robotic platforms. We propose a Dynamic Early-Exit Framework for Robotic Vision-Language-Action Model (DeeR) that automatically adjusts the size of the activated MLLM. DeeR demonstrates significant reductions in computational costs of LLM by 5.2-6.5x and GPU memory of LLM by 2-6x without compromising performance.
  arXiv  Detail & Related papers  (2024-11-04T18:26:08Z)
• Energy-Aware FPGA Implementation of Spiking Neural Network with LIF Neurons [0.5243460995467893]
  Spiking Neural Networks (SNNs) stand out as a cutting-edge solution for TinyML. This paper presents a novel SNN architecture based on the 1st Order Leaky Integrate-and-Fire (LIF) neuron model. A hardware-friendly LIF design is also proposed, and implemented on a Xilinx Artix-7 FPGA.
  arXiv  Detail & Related papers  (2024-11-03T16:42:10Z)
• StreamTinyNet: video streaming analysis with spatial-temporal TinyML [2.5739385355356714]
  Video Streaming Analysis (VSA) is one of the most interesting tasks of TinyML. We present StreamTinyNet, the first TinyML architecture to perform multiple-frame VSA.
  arXiv  Detail & Related papers  (2024-07-22T07:08:03Z)
• Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review [1.049712834719005]
  This review provides an in-depth analysis of the advancements in efficient neural networks and the deployment of deep learning models on ultra-low power microcontrollers. The core of the review centres on efficient neural networks for TinyML. It covers techniques such as model compression, quantization, and low-rank factorization, which optimize neural network architectures for minimal resource utilization. The paper then delves into the deployment of deep learning models on ultra-low power MCUs, addressing challenges such as limited computational capabilities and memory resources.
  arXiv  Detail & Related papers  (2023-11-20T16:20:13Z)
• Signal Detection in MIMO Systems with Hardware Imperfections: Message Passing on Neural Networks [101.59367762974371]
  In this paper, we investigate signal detection in multiple-input-multiple-output (MIMO) communication systems with hardware impairments. It is difficult to train a deep neural network (DNN) with limited pilot signals, hindering its practical applications. We design an efficient message passing based Bayesian signal detector, leveraging the unitary approximate message passing (UAMP) algorithm.
  arXiv  Detail & Related papers  (2022-10-08T04:32:58Z)
• Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms [59.03426963238452]
  We present Real-time Neural MPC, a framework to efficiently integrate large, complex neural network architectures as dynamics models within a model-predictive control pipeline. We show the feasibility of our framework on real-world problems by reducing the positional tracking error by up to 82% when compared to state-of-the-art MPC approaches without neural network dynamics.
  arXiv  Detail & Related papers  (2022-03-15T09:38:15Z)
• FPGA-optimized Hardware acceleration for Spiking Neural Networks [69.49429223251178]
  This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources. It reduces the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.
  arXiv  Detail & Related papers  (2022-01-18T13:59:22Z)
• A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays [66.62377866022221]
  Latent Replay-based Continual Learning (CL) techniques enable online, serverless adaptation in principle. We introduce a HW/SW platform for end-to-end CL based on a 10-core FP32-enabled parallel ultra-low-power processor. Our results show that by combining these techniques, continual learning can be achieved in practice using less than 64MB of memory.
  arXiv  Detail & Related papers  (2021-10-20T11:01:23Z)
• TinyOL: TinyML with Online-Learning on Microcontrollers [7.172671995820974]
  Tiny machine learning (TinyML) is committed to democratizing deep learning for all-pervasive microcontrollers (MCUs) Current TinyML solutions are based on batch/offline settings and support only the neural network's inference on MCUs. We propose a novel system called TinyOL (TinyML with Online-Learning), which enables incremental on-device training on streaming data.
  arXiv  Detail & Related papers  (2021-03-15T11:39:41Z)
• FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation [81.76975488010213]
  Dense optical flow estimation plays a key role in many robotic vision tasks. Current networks often occupy large number of parameters and require heavy computation costs. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
  arXiv  Detail & Related papers  (2021-03-08T03:09:37Z)
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers [18.662026553041937]
  Machine learning on resource constrained microcontrollers (MCUs) promises to drastically expand the application space of the Internet of Things (IoT) TinyML presents severe technical challenges, as deep neural network inference demands a large compute and memory budget. neural architecture search (NAS) promises to help design accurate ML models that meet the tight MCU memory, latency and energy constraints.
  arXiv  Detail & Related papers  (2020-10-21T19:39:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.