Intelligence at the Extreme Edge: A Survey on Reformable TinyML

• URL: http://arxiv.org/abs/2204.00827v1
• Date: Sat, 2 Apr 2022 09:53:36 GMT
• Title: Intelligence at the Extreme Edge: A Survey on Reformable TinyML
• Authors: Visal Rajapakse, Ishan Karunanayake, Nadeem Ahmed
• Abstract summary: We present a survey on reformable TinyML solutions with the proposal of a novel taxonomy for ease of separation. We explore the workflow of TinyML and analyze the identified deployment schemes and the scarcely available benchmarking tools.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid miniaturization of Machine Learning (ML) for low powered processing has opened gateways to provide cognition at the extreme edge (E.g., sensors and actuators). Dubbed Tiny Machine Learning (TinyML), this upsurging research field proposes to democratize the use of Machine Learning (ML) and Deep Learning (DL) on frugal Microcontroller Units (MCUs). MCUs are highly energy-efficient pervasive devices capable of operating with less than a few Milliwatts of power. Nevertheless, many solutions assume that TinyML can only run inference. Despite this, growing interest in TinyML has led to work that makes them reformable, i.e., work that permits TinyML to improve once deployed. In line with this, roadblocks in MCU based solutions in general, such as reduced physical access and long deployment periods of MCUs, deem reformable TinyML to play a significant part in more effective solutions. In this work, we present a survey on reformable TinyML solutions with the proposal of a novel taxonomy for ease of separation. Here, we also discuss the suitability of each hierarchical layer in the taxonomy for allowing reformability. In addition to these, we explore the workflow of TinyML and analyze the identified deployment schemes and the scarcely available benchmarking tools. Furthermore, we discuss how reformable TinyML can impact a few selected industrial areas and discuss the challenges and future directions.

Related papers

• Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity? [3.1471494780647795]
  This paper explores whether Large Language Models (LLMs) could help automate and streamline the TinyML lifecycle. We develop a framework that leverages the natural language processing (NLP) and code generation capabilities of LLMs to reduce development time and lower the barriers to entry for TinyML deployment.
  arXiv  Detail & Related papers  (2025-01-20T22:20:57Z)
• Benchmarking Large and Small MLLMs [71.78055760441256]
  Large multimodal language models (MLLMs) have achieved remarkable advancements in understanding and generating multimodal content. However, their deployment faces significant challenges, including slow inference, high computational cost, and impracticality for on-device applications. Small MLLMs, exemplified by the LLava-series models and Phi-3-Vision, offer promising alternatives with faster inference, reduced deployment costs, and the ability to handle domain-specific scenarios.
  arXiv  Detail & Related papers  (2025-01-04T07:44:49Z)
• LLaVA-KD: A Framework of Distilling Multimodal Large Language Models [70.19607283302712]
  We propose a novel framework to transfer knowledge from l-MLLM to s-MLLM. Specifically, we introduce Multimodal Distillation (MDist) to minimize the divergence between the visual-textual output distributions of l-MLLM and s-MLLM. We also propose a three-stage training scheme to fully exploit the potential of s-MLLM.
  arXiv  Detail & Related papers  (2024-10-21T17:41:28Z)
• Understanding Information Storage and Transfer in Multi-modal Large Language Models [51.20840103605018]
  We study how Multi-modal Large Language Models process information in a factual visual question answering task. Key findings show that these MLLMs rely on self-attention blocks in much earlier layers for information storage. We introduce MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs.
  arXiv  Detail & Related papers  (2024-06-06T16:35:36Z)
• TinySV: Speaker Verification in TinyML with On-device Learning [2.356162747014486]
  This paper introduces a new type of adaptive TinyML solution that can be used in tasks, such as the presented textitTiny Speaker Verification (TinySV) The proposed TinySV solution relies on a two-layer hierarchical TinyML solution comprising Keyword Spotting and Adaptive Speaker Verification module. We evaluate the effectiveness and efficiency of the proposed TinySV solution on a dataset collected expressly for the task and tested the proposed solution on a real-world IoT device.
  arXiv  Detail & Related papers  (2024-06-03T17:27:40Z)
• Exploring Perceptual Limitation of Multimodal Large Language Models [57.567868157293994]
  We quantitatively study the perception of small visual objects in several state-of-the-art MLLMs. We identify four independent factors that can contribute to this limitation. Lower object quality and smaller object size can both independently reduce MLLMs' ability to answer visual questions.
  arXiv  Detail & Related papers  (2024-02-12T03:04:42Z)
• Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE [83.00018517368973]
  Large Language Models (LLMs) can extend their zero-shot capabilities to multimodal learning through instruction tuning. negative conflicts and interference may have a worse impact on performance. We combine the well-known Mixture-of-Experts (MoE) and one of the representative PEFT techniques, i.e., LoRA, designing a novel LLM-based decoder, called LoRA-MoE, for multimodal learning.
  arXiv  Detail & Related papers  (2023-11-05T15:48:29Z)
• TinyMetaFed: Efficient Federated Meta-Learning for TinyML [8.940139322528829]
  We introduce TinyMetaFed, a model-agnostic meta-learning framework suitable for TinyML. TinyMetaFed facilitates collaborative training of a neural network. It offers communication savings and privacy protection through partial local reconstruction and Top-P% selective communication.
  arXiv  Detail & Related papers  (2023-07-13T15:39:26Z)
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models [73.86954509967416]
  Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks. This paper presents the first comprehensive MLLM Evaluation benchmark MME. It measures both perception and cognition abilities on a total of 14 subtasks.
  arXiv  Detail & Related papers  (2023-06-23T09:22:36Z)
• TinyReptile: TinyML with Federated Meta-Learning [9.618821589196624]
  We propose TinyReptile, a simple but efficient algorithm inspired by meta-learning and online learning. We demonstrate TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM.
  arXiv  Detail & Related papers  (2023-04-11T13:11:10Z)
• A review of TinyML [0.0]
  The TinyML concept for embedded machine learning attempts to push such diversity from usual high-end approaches to low-end applications. TinyML is a rapidly expanding interdisciplinary topic at the convergence of machine learning, software, and hardware. This paper explores how TinyML can benefit a few specific industrial fields, its obstacles, and its future scope.
  arXiv  Detail & Related papers  (2022-11-05T06:02:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.