SCOPE: Semantic Coreset with Orthogonal Projection Embeddings for Federated learning

• URL: http://arxiv.org/abs/2603.12976v1
• Date: Fri, 13 Mar 2026 13:24:12 GMT
• Title: SCOPE: Semantic Coreset with Orthogonal Projection Embeddings for Federated learning
• Abstract summary: Current ML approaches either require impractical data aggregation or fail due to class imbalance.<n>We introduce SCOPE, a coreset framework for federated data that filters anomalies and adaptively prunes redundant data to long-tail skew.<n>SCOPE yields competitive global accuracy and robust convergence, all while achieving exceptional efficiency with a 128x to 512x reduction in uplink bandwidth.
• Score: 0.7799711162530713
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Scientific discovery increasingly requires learning on federated datasets, fed by streams from high-resolution instruments, that have extreme class imbalance. Current ML approaches either require impractical data aggregation or fail due to class imbalance. Existing coreset selection methods rely on local heuristics, making them unaware of the global data landscape and prone to sub-optimal and non-representative pruning. To overcome these challenges, we introduce SCOPE (Semantic Coreset using Orthogonal Projection Embeddings for Federated learning), a coreset framework for federated data that filters anomalies and adaptively prunes redundant data to mitigate long-tail skew. By analyzing the latent space distribution, we score each data point using a representation score that measures the reliability of core class features, a diversity score that quantifies the novelty of orthogonal residuals, and a boundary proximity score that indicates similarity to competing classes. Unlike prior methods, SCOPE shares only scalar metrics with a federated server to construct a global consensus, ensuring communication efficiency. Guided by the global consensus, SCOPE dynamically filters local noise and discards redundant samples to counteract global feature skews. Extensive experiments demonstrate that SCOPE yields competitive global accuracy and robust convergence, all while achieving exceptional efficiency with a 128x to 512x reduction in uplink bandwidth, a 7.72x wall-clock acceleration and reduced FLOP and VRAM footprints for local coreset selection.

Related papers

• FedOPAL: One-Shot Federated Learning via Analytic Visual Prompt Tuning [5.979900839748232]
  This paper proposes a framework for efficient collaboration of large models on the edge.<n>FedOPAL adapts the visual prompts as feature prompts, actively correcting the feature distribution of heterogeneous data to a linearly separable space.<n> Experimental results show that FedOPAL not only significantly outperforms the original analytical methods on several benchmarks, but also achieves accuracy comparable to state-of-the-art iterative methods while maintaining zero server-side training costs.
  arXiv  Detail & Related papers  (2026-07-09T11:28:31Z)
• Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approach [10.914852410153921]
  We propose FedEPD, a framework built on a dual decoupling paradigm that separates topological purification from semantic recalibration.<n>FedEPD achieves state-of-the-art performance across diverse long-tailed benchmarks, yielding absolute improvements of up to 4.97% in Accuracy and 5.48% in Macro-F1.
  arXiv  Detail & Related papers  (2026-06-23T07:25:40Z)
• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis [2.298932494750101]
  Federated Learning (FL) enables decentralised model training across distributed clients without requiring data centralisation.<n>We propose FedQuad, a novel method that explicitly enforces minimising intra-class representations while enabling inter-class splits across clients.
  arXiv  Detail & Related papers  (2026-05-08T15:34:41Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Federated Learning Meets LLMs: Feature Extraction From Heterogeneous Clients [0.0]
  Federated learning (FL) enables collaborative model training without sharing raw data.<n>We propose FedLLM-Align, a framework that leverages pre-trained large language models (LLMs) as universal feature extractors.<n>We evaluate FedLLM-Align on coronary heart disease prediction using partitioned datasets with simulated schema divergence.
  arXiv  Detail & Related papers  (2025-09-29T14:06:52Z)
• Coresets from Trajectories: Selecting Data via Correlation of Loss Differences [14.31847187460321]
  Correlation of Loss Differences (CLD) is a scalable metric for coreset selection.<n>On CIFAR-100 and ImageNet-1k, CLD-based coresets typically outperform or closely match state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-27T19:18:39Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Need [83.10178754323955]
  Few-shot learning enables models to generalize from only a few labeled examples.<n>We propose the Unbiased Max-Min Embedding Classification (UMMEC) Method, which addresses the key challenges in few-shot learning.<n>Our method significantly improves classification performance with minimal labeled data, advancing the state-of-the-art in annotatedL.
  arXiv  Detail & Related papers  (2025-03-28T07:23:07Z)
• Exploiting Label Skews in Federated Learning with Model Concatenation [39.38427550571378]
  Federated Learning (FL) has emerged as a promising solution to perform deep learning on different data owners without exchanging raw data. Among different non-IID types, label skews have been challenging and common in image classification and other tasks. We propose FedConcat, a simple and effective approach that degrades these local models as the base of the global model.
  arXiv  Detail & Related papers  (2023-12-11T10:44:52Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Robust Convergence in Federated Learning through Label-wise Clustering [6.693651193181458]
  Non-IID dataset and heterogeneous environment of the local clients are regarded as a major issue in Federated Learning (FL) We propose a novel Label-wise clustering algorithm that guarantees the trainability among geographically heterogeneous local clients. Our paper shows that proposed Label-wise clustering demonstrates prompt and robust convergence compared to other FL algorithms.
  arXiv  Detail & Related papers  (2021-12-28T18:13:09Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.