ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning

• URL: http://arxiv.org/abs/2505.16850v1
• Date: Thu, 22 May 2025 16:11:38 GMT
• Title: ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning
• Authors: Tajamul Ashraf, Mohammed Mohsen Peerzada, Moloud Abdar, Yutong Xie, Yuyin Zhou, Xiaofeng Liu, Iqra Altaf Gillani, Janibul Bashir,
• Abstract summary: We introduce a unified framework for analyzing federated learning through three foundational dimensions: Adaptation, Trust, and Reasoning.<n>ATR-Bench lays the groundwork for a systematic and holistic evaluation of federated learning with real-world relevance.
• Score: 21.099779419619345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) has emerged as a promising paradigm for collaborative model training while preserving data privacy across decentralized participants. As FL adoption grows, numerous techniques have been proposed to tackle its practical challenges. However, the lack of standardized evaluation across key dimensions hampers systematic progress and fair comparison of FL methods. In this work, we introduce ATR-Bench, a unified framework for analyzing federated learning through three foundational dimensions: Adaptation, Trust, and Reasoning. We provide an in-depth examination of the conceptual foundations, task formulations, and open research challenges associated with each theme. We have extensively benchmarked representative methods and datasets for adaptation to heterogeneous clients and trustworthiness in adversarial or unreliable environments. Due to the lack of reliable metrics and models for reasoning in FL, we only provide literature-driven insights for this dimension. ATR-Bench lays the groundwork for a systematic and holistic evaluation of federated learning with real-world relevance. We will make our complete codebase publicly accessible and a curated repository that continuously tracks new developments and research in the FL literature.

Related papers

• OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics [101.78963920333342]
  We introduce OpenUnlearning, a standardized framework for benchmarking large language models (LLMs) unlearning methods and metrics.<n>OpenUnlearning integrates 9 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks.<n>We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite.
  arXiv  Detail & Related papers  (2025-06-14T20:16:37Z)
• A Survey on Federated Fine-tuning of Large Language Models [21.559090305077273]
  FedLLM offers a promising avenue for collaborative model adaptation while preserving data privacy.<n>To rigorously evaluate FedLLM, we undertake a thorough review of existing fine-tuning datasets and evaluation benchmarks.<n>We identify critical open challenges and outline promising research directions to foster future advancements in FedLLM.
  arXiv  Detail & Related papers  (2025-03-15T06:52:10Z)
• FedEGG: Federated Learning with Explicit Global Guidance [90.04705121816185]
  Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature.<n>Existing methods help address these challenges via optimization-based client constraints, adaptive client selection, or the use of pre-trained models or synthetic data.<n>We present bftextFedEGG, a new FL algorithm that constructs a global guiding task using a well-defined, easy-to-converge learning task.
  arXiv  Detail & Related papers  (2024-04-18T04:25:21Z)
• A Survey on Federated Unlearning: Challenges and Opportunities [32.0365189539138]
  This SoK paper aims to take a deep look at the emphfederated unlearning literature, with the goal of identifying research trends and challenges in this emerging field.
  arXiv  Detail & Related papers  (2024-03-04T19:35:08Z)
• Exploring Federated Unlearning: Review, Comparison, and Insights [101.64910079905566]
  federated unlearning enables the selective removal of data from models trained in federated systems.<n>This paper examines existing federated unlearning approaches, examining their algorithmic efficiency, impact on model accuracy, and effectiveness in preserving privacy.<n>We propose the OpenFederatedUnlearning framework, a unified benchmark for evaluating federated unlearning methods.
  arXiv  Detail & Related papers  (2023-10-30T01:34:33Z)
• A Survey for Federated Learning Evaluations: Goals and Measures [26.120949005265345]
  Federated learning (FL) is a novel paradigm for privacy-preserving machine learning. evaluating FL is challenging due to its interdisciplinary nature and diverse goals, such as utility, efficiency, and security. We introduce FedEval, an open-source platform that provides a standardized and comprehensive evaluation framework for FL algorithms.
  arXiv  Detail & Related papers  (2023-08-23T00:17:51Z)
• Deep Equilibrium Models Meet Federated Learning [71.57324258813675]
  This study explores the problem of Federated Learning (FL) by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning.
  arXiv  Detail & Related papers  (2023-05-29T22:51:40Z)
• Trustworthy Federated Learning: A Survey [0.5089078998562185]
  Federated Learning (FL) has emerged as a significant advancement in the field of Artificial Intelligence (AI) We provide an extensive overview of the current state of Trustworthy FL, exploring existing solutions and well-defined pillars relevant to Trustworthy. We propose a taxonomy that encompasses three main pillars: Interpretability, Fairness, and Security & Privacy.
  arXiv  Detail & Related papers  (2023-05-19T09:11:26Z)
• Towards Verifiable Federated Learning [15.758657927386263]
  Federated learning (FL) is an emerging paradigm of collaborative machine learning that preserves user privacy while building powerful models. Due to the nature of open participation by self-interested entities, FL needs to guard against potential misbehaviours by legitimate FL participants. Verifiable federated learning has become an emerging topic of research that has attracted significant interest from the academia and the industry alike.
  arXiv  Detail & Related papers  (2022-02-15T09:52:25Z)
• RoFL: Attestable Robustness for Secure Federated Learning [59.63865074749391]
  Federated Learning allows a large number of clients to train a joint model without the need to share their private data. To ensure the confidentiality of the client updates, Federated Learning systems employ secure aggregation. We present RoFL, a secure Federated Learning system that improves robustness against malicious clients.
  arXiv  Detail & Related papers  (2021-07-07T15:42:49Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.