Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning

• URL: http://arxiv.org/abs/2410.18378v1
• Date: Thu, 24 Oct 2024 02:38:09 GMT
• Title: Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning
• Authors: Chen Gong, Zhenzhe Zheng, Fan Wu, Xiaofeng Jia, Guihai Chen,
• Abstract summary: We explore the potential of leveraging abundant cloud-side data to enrich scarce on-device data, and propose a private, efficient and effective data enrichment framework Delta. Specifically, Delta first introduces a directory dataset to decompose the data enrichment problem into device-side and cloud-side sub-problems without sharing sensitive data. Next, Delta proposes a soft data matching strategy to effectively solve the device-side sub-problem with sparse user data, and an optimal data sampling scheme for cloud server to retrieve the most suitable dataset for enrichment with low computational complexity.
• Score: 43.297365394830294
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In modern mobile applications, users frequently encounter various new contexts, necessitating on-device continual learning (CL) to ensure consistent model performance. While existing research predominantly focused on developing lightweight CL frameworks, we identify that data scarcity is a critical bottleneck for on-device CL. In this work, we explore the potential of leveraging abundant cloud-side data to enrich scarce on-device data, and propose a private, efficient and effective data enrichment framework Delta. Specifically, Delta first introduces a directory dataset to decompose the data enrichment problem into device-side and cloud-side sub-problems without sharing sensitive data. Next, Delta proposes a soft data matching strategy to effectively solve the device-side sub-problem with sparse user data, and an optimal data sampling scheme for cloud server to retrieve the most suitable dataset for enrichment with low computational complexity. Further, Delta refines the data sampling scheme by jointly considering the impact of enriched data on both new and past contexts, mitigating the catastrophic forgetting issue from a new aspect. Comprehensive experiments across four typical mobile computing tasks with varied data modalities demonstrate that Delta could enhance the overall model accuracy by an average of 15.1%, 12.4%, 1.1% and 5.6% for visual, IMU, audio and textual tasks compared with few-shot CL, and consistently reduce the communication costs by over 90% compared to federated CL.

Related papers

• Studying the Role of Synthetic Data for Machine Learning-based Wireless Networks Traffic Forecasting [1.1699027359021665]
  This paper proposes a novel method to generate synthetic data, based on first-order auto-regressive noise statistics, for large-scale Wi-Fi deployments.<n> Experimental results show that ML models trained on synthetic data achieve Mean Absolute Error (MAE) values within 10 to 15 of those obtained using real data.<n>When generalization is required, synthetic-data-trained models improve prediction accuracy by up to 50 percent compared to real-data-trained baselines.
  arXiv  Detail & Related papers  (2026-01-12T15:27:55Z)
• Dynamic Clustering for Personalized Federated Learning on Heterogeneous Edge Devices [10.51330114955586]
  Federated Learning (FL) enables edge devices to collaboratively learn a global model.<n>We propose a dynamic clustering algorithm for personalized federated learning system (DC-PFL)<n>We show that DC-PFL significantly reduces total training time and improves model accuracy compared to baselines.
  arXiv  Detail & Related papers  (2025-08-03T04:19:22Z)
• FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation [21.910537847630067]
  Residual connection has been extensively studied and widely applied at the model architecture level.<n>We introduce the concept of Data Residual Matching for the first time, leveraging data-level skip connections to facilitate data generation and mitigate data information vanishing.
  arXiv  Detail & Related papers  (2025-06-30T17:59:34Z)
• Less is More: Adaptive Coverage for Synthetic Training Data [20.136698279893857]
  This study introduces a novel sampling algorithm, based on the maximum coverage problem, to select a representative subset from a synthetically generated dataset. Our results demonstrate that training a classifier on this contextually sampled subset achieves superior performance compared to training on the entire dataset.
  arXiv  Detail & Related papers  (2025-04-20T06:45:16Z)
• One-shot Federated Learning via Synthetic Distiller-Distillate Communication [63.89557765137003]
  One-shot Federated learning (FL) is a powerful technology facilitating collaborative training of machine learning models in a single round of communication. We propose FedSD2C, a novel and practical one-shot FL framework designed to address these challenges.
  arXiv  Detail & Related papers  (2024-12-06T17:05:34Z)
• A CLIP-Powered Framework for Robust and Generalizable Data Selection [51.46695086779598]
  Real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance. Data selection has shown promise in identifying the most representative samples from the entire dataset. We propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection.
  arXiv  Detail & Related papers  (2024-10-15T03:00:58Z)
• Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data [9.045647166114916]
  Federated Learning (FL) is a promising paradigm for decentralized and collaborative model training. FL struggles with a significant performance reduction and poor convergence when confronted with Non-Independent and Identically Distributed (Non-IID) data distributions. We introduce Gen-FedSD, a novel approach that harnesses the powerful capability of state-of-the-art text-to-image foundation models.
  arXiv  Detail & Related papers  (2024-05-13T16:57:48Z)
• Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents [43.282328554697564]
  We propose a highly efficient FL dataset distillation framework on the server side. Unlike previous strategies, our technique enables the server to leverage prior knowledge from pre-trained deep generative models. Our framework converges faster than the baselines because rather than the server trains on several sets of heterogeneous data distributions, it trains on a multi-modal distribution.
  arXiv  Detail & Related papers  (2023-12-03T23:30:48Z)
• Multi-Resolution Diffusion for Privacy-Sensitive Recommender Systems [2.812395851874055]
  We introduce a Score-based Diffusion Recommendation Module (SDRM), which captures the intricate patterns of real-world datasets required for training highly accurate recommender systems. SDRM allows for the generation of synthetic data that can replace existing datasets to preserve user privacy, or augment existing datasets to address excessive data sparsity. Our method outperforms competing baselines such as generative adversarial networks, variational autoencoders, and recently proposed diffusion models in synthesizing various datasets to replace or augment the original data by an average improvement of 4.30% in Recall@k and 4.65% in NDCG@k.
  arXiv  Detail & Related papers  (2023-11-06T19:52:55Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• Scaling Data Generation in Vision-and-Language Navigation [116.95534559103788]
  We propose an effective paradigm for generating large-scale data for learning. We apply 1200+ photo-realistic environments from HM3D and Gibson datasets and synthesizes 4.9 million instruction trajectory pairs. Thanks to our large-scale dataset, the performance of an existing agent can be pushed up (+11% absolute with regard to previous SoTA) to a significantly new best of 80% single-run success rate on the R2R test split by simple imitation learning.
  arXiv  Detail & Related papers  (2023-07-28T16:03:28Z)
• Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
  Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices. The impact of on-device storage on the performance of FL is still not explored. In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
  arXiv  Detail & Related papers  (2022-09-01T03:27:33Z)
• DataPerf: Benchmarks for Data-Centric AI Development [81.03754002516862]
  DataPerf is a community-led benchmark suite for evaluating ML datasets and data-centric algorithms. We provide an open, online platform with multiple rounds of challenges to support this iterative development. The benchmarks, online evaluation platform, and baseline implementations are open source.
  arXiv  Detail & Related papers  (2022-07-20T17:47:54Z)
• Federated Visual Classification with Real-World Data Distribution [9.564468846277366]
  We characterize the effect real-world data distributions have on distributed learning, using as a benchmark the standard Federated Averaging (FedAvg) algorithm. We introduce two new large-scale datasets for species and landmark classification, with realistic per-user data splits. We also develop two new algorithms (FedVC, FedIR) that intelligently resample and reweight over the client pool, bringing large improvements in accuracy and stability in training.
  arXiv  Detail & Related papers  (2020-03-18T07:55:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.