A Universal Metric of Dataset Similarity for Cross-silo Federated Learning

• URL: http://arxiv.org/abs/2404.18773v1
• Date: Mon, 29 Apr 2024 15:08:24 GMT
• Title: A Universal Metric of Dataset Similarity for Cross-silo Federated Learning
• Authors: Ahmed Elhussein, Gamze Gursoy,
• Abstract summary: Federated learning is increasingly used in domains such as healthcare to facilitate model training without data-sharing. In this paper, we propose a novel metric for assessing dataset similarity. We show that our metric shows a robust and interpretable relationship with model performance and can be calculated in privacy-preserving manner.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning is increasingly used in domains such as healthcare to facilitate collaborative model training without data-sharing. However, datasets located in different sites are often non-identically distributed, leading to degradation of model performance in FL. Most existing methods for assessing these distribution shifts are limited by being dataset or task-specific. Moreover, these metrics can only be calculated by exchanging data, a practice restricted in many FL scenarios. To address these challenges, we propose a novel metric for assessing dataset similarity. Our metric exhibits several desirable properties for FL: it is dataset-agnostic, is calculated in a privacy-preserving manner, and is computationally efficient, requiring no model training. In this paper, we first establish a theoretical connection between our metric and training dynamics in FL. Next, we extensively evaluate our metric on a range of datasets including synthetic, benchmark, and medical imaging datasets. We demonstrate that our metric shows a robust and interpretable relationship with model performance and can be calculated in privacy-preserving manner. As the first federated dataset similarity metric, we believe this metric can better facilitate successful collaborations between sites.

Related papers

• Piecewise-Linear Manifolds for Deep Metric Learning [8.670873561640903]
  Unsupervised deep metric learning focuses on learning a semantic representation space using only unlabeled data. We propose to model the high-dimensional data manifold using a piecewise-linear approximation, with each low-dimensional linear piece approximating the data manifold in a small neighborhood of a point. We empirically show that this similarity estimate correlates better with the ground truth than the similarity estimates of current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-03-22T06:22:20Z)
• SoK: Challenges and Opportunities in Federated Unlearning [32.0365189539138]
  This SoK paper aims to take a deep look at the emphfederated unlearning literature, with the goal of identifying research trends and challenges in this emerging field.
  arXiv  Detail & Related papers  (2024-03-04T19:35:08Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Tackling Computational Heterogeneity in FL: A Few Theoretical Insights [68.8204255655161]
  We introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneous data. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.
  arXiv  Detail & Related papers  (2023-07-12T16:28:21Z)
• FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings [51.09574369310246]
  Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models. We propose a novel cross-silo dataset suite focused on healthcare, FLamby, to bridge the gap between theory and practice of cross-silo FL. Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research.
  arXiv  Detail & Related papers  (2022-10-10T12:17:30Z)
• FedDAR: Federated Domain-Aware Representation Learning [14.174833360938806]
  Cross-silo Federated learning (FL) has become a promising tool in machine learning applications for healthcare. We propose a novel method, FedDAR, which learns a domain shared representation and domain-wise personalized prediction heads.
  arXiv  Detail & Related papers  (2022-09-08T19:18:59Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Learning Similarity Metrics for Numerical Simulations [29.39625644221578]
  We propose a neural network-based approach that computes a stable and generalizing metric (LSiM) to compare data from a variety of numerical simulation sources. Our method employs a Siamese network architecture that is motivated by the mathematical properties of a metric.
  arXiv  Detail & Related papers  (2020-02-18T20:11:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.