From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning

• URL: http://arxiv.org/abs/2606.10595v1
• Date: Tue, 09 Jun 2026 09:00:04 GMT
• Title: From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning
• Abstract summary: Federated Learning (FL) has emerged as a promising solution for data hunger in centralized learning.<n>FL enables privacy with multiple clients to train a shared-task model collaboratively without exposing their local data.<n>While being a key component in any learning system, data is also a primary source of vulnerabilities and challenges.
• Score: 35.26793524140591
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) has emerged as a promising solution for data hunger in centralized learning. This paradigm enables privacy with multiple clients to train a shared-task model collaboratively without exposing their local data. While being a key component in any learning system, data is also a primary source of vulnerabilities and challenges, and a major determinant of a stable and well-converged training. Existing FL reviews describe general foundations, security practices, opportunities, challenges, and applications, without delving into diverse aspects of data and considering problems from the data perspective. They rarely provide a data-lens synthesis that links concrete data properties, split protocols, and defenses to convergence speed and stability. This survey fills that gap with three advances. First, we analyze non-IID into measurable traits and rank their influence on convergence as strong, medium, or light, explaining the mechanisms behind each and reconciling evidence across images, texts, and graphs. Second, we connect experimental splitting practices to the real phenomena they emulate, expose the artifacts they introduce, and show how those artifacts affect target accuracy. Third, we analyze how data-related vulnerabilities and their proposed defenses affect convergence, reporting performance under clean and adversarial conditions to make the convergence-robustness trade-off explicit. To our knowledge, this is the first survey to provide a complete understanding of data-related challenges that govern FL. With clear takeaways distilled for each concern, our work serves as actionable guidance, helping practitioners design their system with predictable convergence and stability.

Related papers

• Evian: Towards Explainable Visual Instruction-tuning Data Auditing [14.93566912726999]
  We introduce a novel "Decomposition-then-Evaluation" paradigm that breaks model responses into constituent cognitive components.<n>We instantiate this paradigm via EVIAN, an automated framework that evaluates these components along the axes of Image-Text Consistency, Logical Coherence, and Factual Accuracy.
  arXiv  Detail & Related papers  (2026-04-22T13:28:27Z)
• Data Fusion for Partial Identification of Causal Effects [62.56890808004615]
  We propose a novel partial identification framework that enables researchers to answer key questions.<n>Is the causal effect positive or negative? and How severe must assumption violations be to overturn this conclusion?<n>We apply our framework to the Project STAR study, which investigates the effect of classroom size on students' third-grade standardized test performance.
  arXiv  Detail & Related papers  (2025-05-30T07:13:01Z)
• FedSKC: Federated Learning with Non-IID Data via Structural Knowledge Collaboration [43.25824181502647]
  Key idea of FedSKC is to extract and transfer domain preferences from interclient data distributions.<n>FedSKC comprises three components: contrastive learning, global discrepancy aggregation, and global period review.
  arXiv  Detail & Related papers  (2025-05-25T05:24:49Z)
• A Thorough Assessment of the Non-IID Data Impact in Federated Learning [1.89791888476598]
  Decentralized machine learning (FL) allows collaborative machine learning (ML) training among decentralized clients' information, ensuring data privacy.<n>The decentralized nature of FL deals with non-independent and identically distributed (non-IID) data.<n>Despite its importance, experimental studies systematically addressing all types of data heterogeneity (ak.a. non-IIDness) remain scarce.<n>We aim to fill this gap by assessing and quantifying the non-IID effect through a thorough empirical analysis.
  arXiv  Detail & Related papers  (2025-03-21T11:53:36Z)
• Footprints of Data in a Classifier: Understanding the Privacy Risks and Solution Strategies [0.9208007322096533]
  Article 17 of the General Data Protection Regulation (Right Erasure) requires data to be permanently removed from a system to prevent potential compromise.<n>One such issue arises from the residual footprints of training data embedded within predictive models.<n>This study examines how two fundamental aspects of classifier systems - training quality and classifier training methodology - contribute to privacy vulnerabilities.
  arXiv  Detail & Related papers  (2024-07-02T13:56:37Z)
• How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective [29.924482732745954]
  Existing approaches assume data points to-be-forgotten are independent, ignoring their inter-connectivity.<n>We propose PISTOL, a method for compiling structural datasets.
  arXiv  Detail & Related papers  (2024-06-24T17:22:36Z)
• DAGnosis: Localized Identification of Data Inconsistencies using Structures [73.39285449012255]
  Identification and appropriate handling of inconsistencies in data at deployment time is crucial to reliably use machine learning models. We use directed acyclic graphs (DAGs) to encode the training set's features probability distribution and independencies as a structure. Our method, called DAGnosis, leverages these structural interactions to bring valuable and insightful data-centric conclusions.
  arXiv  Detail & Related papers  (2024-02-26T11:29:16Z)
• FLASH: Federated Learning Across Simultaneous Heterogeneities [55.0981921695672]
  FLASH(Federated Learning Across Simultaneous Heterogeneities) is a lightweight and flexible client selection algorithm.<n>It outperforms state-of-the-art FL frameworks under extensive sources of Heterogeneities.<n>It achieves substantial and consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:04:39Z)
• Towards Understanding Adversarial Transferability in Federated Learning [14.417827137513369]
  A group of malicious clients has impacted the model during training by disguising their identities and acting as benign clients but later switching to an adversarial role. This type of attack is subtle and hard to detect because these clients initially appear to be benign. We empirically show that the proposed attack imposes a high security risk to current FL systems.
  arXiv  Detail & Related papers  (2023-10-01T08:35:46Z)
• Federated Causal Discovery [74.37739054932733]
  This paper develops a gradient-based learning framework named DAG-Shared Federated Causal Discovery (DS-FCD) It can learn the causal graph without directly touching local data and naturally handle the data heterogeneity. Extensive experiments on both synthetic and real-world datasets verify the efficacy of the proposed method.
  arXiv  Detail & Related papers  (2021-12-07T08:04:12Z)
• A Principled Approach to Data Valuation for Federated Learning [73.19984041333599]
  Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value.
  arXiv  Detail & Related papers  (2020-09-14T04:37:54Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.