MetaCLBench: Meta Continual Learning Benchmark on Resource-Constrained Edge Devices

• URL: http://arxiv.org/abs/2504.00174v1
• Date: Mon, 31 Mar 2025 19:31:49 GMT
• Title: MetaCLBench: Meta Continual Learning Benchmark on Resource-Constrained Edge Devices
• Authors: Sijia Li, Young D. Kwon, Lik-Hang Lee, Pan Hui,
• Abstract summary: We develop an end-to-end Meta-CL benchmark framework for edge devices to evaluate system overheads and investigate trade-offs among performance, computational costs, and memory requirements.<n>Our results reveal that while many Meta-CL methods enable to learn new classes for both image and audio modalities, they impose significant computational and memory costs on edge devices.<n>Also, we find that pre-training and meta-training procedures based on source data before deployment improve Meta-CL performance.
• Score: 17.800367605774863
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-Continual Learning (Meta-CL) has emerged as a promising approach to minimize manual labeling efforts and system resource requirements by enabling Continual Learning (CL) with limited labeled samples. However, while existing methods have shown success in image-based tasks, their effectiveness remains unexplored for sequential time-series data from sensor systems, particularly audio inputs. To address this gap, we conduct a comprehensive benchmark study evaluating six representative Meta-CL approaches using three network architectures on five datasets from both image and audio modalities. We develop MetaCLBench, an end-to-end Meta-CL benchmark framework for edge devices to evaluate system overheads and investigate trade-offs among performance, computational costs, and memory requirements across various Meta-CL methods. Our results reveal that while many Meta-CL methods enable to learn new classes for both image and audio modalities, they impose significant computational and memory costs on edge devices. Also, we find that pre-training and meta-training procedures based on source data before deployment improve Meta-CL performance. Finally, to facilitate further research, we provide practical guidelines for researchers and machine learning practitioners implementing Meta-CL on resource-constrained environments and make our benchmark framework and tools publicly available, enabling fair evaluation across both accuracy and system-level metrics.

Related papers

• MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning [27.66874423453976]
  We introduce MEAL, the first benchmark tailored for continual multi-agent reinforcement learning (CMARL)<n>Existing CL benchmarks run environments on the CPU, leading to computational bottlenecks and limiting the length of task sequences.<n>MEAL leverages JAX for GPU acceleration, enabling continual learning across sequences of 100 tasks on a standard desktop PC in a few hours.
  arXiv  Detail & Related papers  (2025-06-17T21:50:04Z)
• OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics [101.78963920333342]
  We introduce OpenUnlearning, a standardized framework for benchmarking large language models (LLMs) unlearning methods and metrics.<n>OpenUnlearning integrates 9 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks.<n>We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite.
  arXiv  Detail & Related papers  (2025-06-14T20:16:37Z)
• Recent Advances of Multimodal Continual Learning: A Comprehensive Survey [64.82070119713207]
  We present the first comprehensive survey on multimodal continual learning methods. We categorize existing MMCL methods into four categories, i.e., regularization-based, architecture-based, replay-based, and prompt-based. We discuss several promising future directions for investigation and development.
  arXiv  Detail & Related papers  (2024-10-07T13:10:40Z)
• Meta-RTL: Reinforcement-Based Meta-Transfer Learning for Low-Resource Commonsense Reasoning [61.8360232713375]
  We propose a reinforcement-based multi-source meta-transfer learning framework (Meta-RTL) for low-resource commonsense reasoning. We present a reinforcement-based approach to dynamically estimating source task weights that measure the contribution of the corresponding tasks to the target task in the meta-transfer learning. Experimental results demonstrate that Meta-RTL substantially outperforms strong baselines and previous task selection strategies.
  arXiv  Detail & Related papers  (2024-09-27T18:22:22Z)
• LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms [17.031135153343502]
  Continual Learning (CL) allows applications such as user personalization and household robots to learn on the fly and adapt to context. LifeLearner is a hardware-aware meta learning system that drastically optimize system resources. LifeLearner achieves near-optimal CL performance, falling short by only 2.8% on accuracy compared to an Oracle baseline.
  arXiv  Detail & Related papers  (2023-11-19T20:39:35Z)
• DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning [55.733193075728096]
  We propose a meta-knowledge informed meta-learning (MKIML) framework to improve meta-learning. We preliminarily integrate meta-knowledge into meta-objective via using an appropriate meta-regularization (MR) objective. The proposed DAC-MR is hopeful to learn well-performing meta-models from training tasks with noisy, sparse or unavailable meta-data.
  arXiv  Detail & Related papers  (2023-05-13T11:01:47Z)
• Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
  Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios. By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels. Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
  arXiv  Detail & Related papers  (2021-06-04T20:15:34Z)
• Information-Theoretic Generalization Bounds for Meta-Learning and Applications [42.275148861039895]
  Key performance measure for meta-learning is the meta-generalization gap. This paper presents novel information-theoretic upper bounds on the meta-generalization gap.
  arXiv  Detail & Related papers  (2020-05-09T05:48:01Z)
• Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning [79.25478727351604]
  We explore a simple process: meta-learning over a whole-classification pre-trained model on its evaluation metric. We observe this simple method achieves competitive performance to state-of-the-art methods on standard benchmarks.
  arXiv  Detail & Related papers  (2020-03-09T20:06:36Z)
• Structured Prediction for Conditional Meta-Learning [44.30857707980074]
  We propose a new perspective on conditional meta-learning via structured prediction. We derive task-adaptive structured meta-learning (TASML), a principled framework that yields task-specific objective functions. Empirically, we show that TASML improves the performance of existing meta-learning models, and outperforms the state-of-the-art on benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-20T15:24:15Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.