On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontroller

• URL: http://arxiv.org/abs/2604.23012v1
• Date: Fri, 24 Apr 2026 20:59:16 GMT
• Title: On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontroller
• Abstract summary: This paper presents an end-to-end on-device vision machine learning pipeline, executing entirely on a microcontroller-class device costing $15-40 USD.<n>Running on the Seeed Studio ESP32-S3 XI Kit MLAO (8 MB PSRAM), the firmware achieves three-class 64x64 image classification in approximately 9 minutes per training run, with real-time inference at 6.3 FPS.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a complete, end-to-end on-device vision machine learning pipeline, comprising data acquisition, two-layer CNN training with Adam optimization, and real-time inference, executing entirely on a microcontroller-class device costing $15-40 USD. Unlike cloud-based workflows that require external infrastructure and conceal the computational pipeline from the practitioner, this system implements every step of the core ML lifecycle in approximately 1,750 lines of readable C++ that compiles in under one minute using the Arduino IDE, with no external ML dependencies. Running on the Seeed Studio ESP32-S3 XIAO ML Kit (8 MB PSRAM), the firmware achieves three-class 64x64 image classification in approximately 9 minutes per training run, with real-time inference at 6.3 FPS. Key contributions include: correct batch-level gradient accumulation; pre-computed resize lookup tables for inference; dual-format weight export for SD-free baked-in deployment; a three-tier weight priority system (SD binary > baked-in header > He-initialization) resolved automatically at boot; a single-constant network reconfiguration interface; and PSRAM-aware memory management suited to microcontroller constraints. All source code and reference datasets are released under the MIT License at https://github.com/webmcu-ai/on-device-vision-ai

Related papers

• Deep Microcompression: Structured Pruning and Bit-packed Quantization for Microcontrollers [0.0]
  This paper introduces Deep Microcompression (DMC), a hardware-aware pipeline for deep learning inference on bare-metal microcontrollers.<n>DMC integrates structured pruning, quantization-aware training, and fixed-length bit-packing to achieve a 55.8times weight compression ratio on LeNet-5 (98.77% accuracy)<n>DMC enables the first documented deployment of a standard CNN on the ATmega328P, a device constrained to 2KB, previously considered infeasible for CNN inference.
  arXiv  Detail & Related papers  (2026-09-04T12:38:07Z)
• OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research [1.1654568410166084]
  OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models.<n>The package includes 27 presets across nine familiar model families and documentation that progresses from LM fundamentals to architecture research.
  arXiv  Detail & Related papers  (2026-07-18T06:55:18Z)
• ProCUA-SFT Technical Report [80.97543110323542]
  Training computer-use agents (CUAs) requires large-scale, diverse trajectory data collected in full desktop environments.<n>We present ProCUA-SFT, a dataset of 3.1M step-level SFT samples distilled from 93K synthetic trajectories.<n>Fine-tuning UI-TARS 7B on ProCUA-SFT for one epoch yields 45.0% on OSWorld -- an 18.7 percentage-point improvement over the base model.
  arXiv  Detail & Related papers  (2026-06-15T22:04:11Z)
• Enabling Performant and Flexible Model-Internal Observability for LLM Inference [6.153611368890026]
  DMI-Lib is a high-speed deep model inspector that treats internal observability as a first-class systems primitive.<n>We show that DMI-Lib incurs only 0.4%--6.8% overhead in offline batch inference and an average of 6% in moderate online serving.
  arXiv  Detail & Related papers  (2026-05-11T18:01:36Z)
• WebSerial Vision Training for Microcontrollers: A Browser-Based Companion to On-Device CNN Training [0.0]
  webmcu-vision-web is a browser-based application for end-to-end TinyML vision model training and deployment on the Seeed Studio XIAO ESP32-S3 Sense.<n>It provides a private, fully local machine learning pipeline, from flashing through image, CNN training, weight export, and live activation visualization.
  arXiv  Detail & Related papers  (2026-04-20T05:25:41Z)
• In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD [51.04126395480625]
  Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. This work offers a solution by simplifying this coupling and enabling in situ training and inference on heterogeneous clusters.
  arXiv  Detail & Related papers  (2023-06-22T14:07:54Z)
• Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers [15.37318446043671]
  This paper introduces a novel reduced precision optimization technique for On-Device Learning (ODL) primitives on MCU-class devices. Our approach results more than two orders of magnitude faster than existing ODL software frameworks for single-core MCUs.
  arXiv  Detail & Related papers  (2023-05-30T16:14:16Z)
• HDCC: A Hyperdimensional Computing compiler for classification on embedded systems and high-performance computing [58.720142291102135]
  This work introduces the name compiler, the first open-source compiler that translates high-level descriptions of HDC classification methods into optimized C code. name is designed like a modern compiler, featuring an intuitive and descriptive input language, an intermediate representation (IR), and a retargetable backend. To substantiate these claims, we conducted experiments with HDCC on several of the most popular datasets in the HDC literature.
  arXiv  Detail & Related papers  (2023-04-24T19:16:03Z)
• An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System [9.429605859159023]
  Training machine learning (ML) algorithms is a computationally intensive process, which is frequently memory-bound. Memory-centric computing systems, with processing-in-memory capabilities, can alleviate this data movement bottleneck. We implement several representative classic ML algorithms on a real-world general-purpose PIM architecture.
  arXiv  Detail & Related papers  (2022-07-16T09:39:53Z)
• On-Device Training Under 256KB Memory [62.95579393237751]
  We propose an algorithm-system co-design framework to make on-device training possible with only 256KB of memory. Our framework is the first solution to enable tiny on-device training of convolutional neural networks under 256KB and 1MB Flash.
  arXiv  Detail & Related papers  (2022-06-30T17:59:08Z)
• A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays [66.62377866022221]
  Latent Replay-based Continual Learning (CL) techniques enable online, serverless adaptation in principle. We introduce a HW/SW platform for end-to-end CL based on a 10-core FP32-enabled parallel ultra-low-power processor. Our results show that by combining these techniques, continual learning can be achieved in practice using less than 64MB of memory.
  arXiv  Detail & Related papers  (2021-10-20T11:01:23Z)
• MCUNet: Tiny Deep Learning on IoT Devices [62.752899523628066]
  We propose a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine) TinyNAS adopts a two-stage neural architecture search approach that first optimize the search space to fit the resource constraints, then specializes the network architecture in the optimized search space. TinyEngine adapts the memory scheduling according to the overall network topology rather than layer-wise optimization, reducing the memory usage by 4.8x.
  arXiv  Detail & Related papers  (2020-07-20T17:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.