Prediction-space knowledge markets for communication-efficient federated learning on multimedia tasks

• URL: http://arxiv.org/abs/2512.00841v1
• Date: Sun, 30 Nov 2025 11:24:19 GMT
• Title: Prediction-space knowledge markets for communication-efficient federated learning on multimedia tasks
• Abstract summary: Federated learning (FL) enables collaborative training over distributed multimedia data.<n>We propose KTA v2, a prediction-space knowledge trading market for FL.<n>Experiments on FEMNIST, CIFAR-10 and AG News show that KTA v2 consistently outperforms a local-only baseline and strong parameter-based methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) enables collaborative training over distributed multimedia data but suffers acutely from statistical heterogeneity and communication constraints, especially when clients deploy large models. Classic parameter-averaging methods such as FedAvg transmit full model weights and can diverge under nonindependent and identically distributed (non-IID) data. We propose KTA v2, a prediction-space knowledge trading market for FL. Each round, clients locally train on their private data, then share only logits on a small public reference set. The server constructs a client-client similarity graph in prediction space, combines it with reference-set accuracy to form per-client teacher ensembles, and sends back personalized soft targets for a second-stage distillation update. This two-stage procedure can be interpreted as approximate block-coordinate descent on a unified objective with prediction-space regularization. Experiments on FEMNIST, CIFAR-10 and AG News show that, under comparable or much lower communication budgets, KTA v2 consistently outperforms a local-only baseline and strong parameter-based methods (FedAvg, FedProx), and substantially improves over a FedMD-style global teacher. On CIFAR-10 with ResNet-18, KTA v2 reaches 57.7% test accuracy using approximately 1/1100 of FedAvg's communication, while on AG News it attains 89.3% accuracy with approximately 1/300 of FedAvg's traffic.

Related papers

• FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning [6.693752118202639]
  Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data.<n>In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server.<n>We propose FedCC, a simple and effective algorithm for mitigating client misclassification.
  arXiv  Detail & Related papers  (2026-08-24T09:34:26Z)
• Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search [7.461859467262201]
  Federated Learning (FL) enables collaborative model training without centralizing data.<n>Existing FL approaches mitigate these challenges through improved aggregation, personalization, or knowledge distillation.<n>This work introduces FedKDNAS, a distillation-driven FL framework that combines client-side neural architecture selection with distillation of server-coordinated knowledge.
  arXiv  Detail & Related papers  (2026-05-20T15:50:49Z)
• FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning [58.80511162314259]
  We propose FedAttr, a new client-level attribution protocol for federated learning.<n>FedAttr identifies which clients trained on watermarked data via a paired-subset-difference mechanism.<n>It achieves 100% TPR and 0% FPR, outperforming all baselines by at least 44.4% in TPR or 19.1% in FPR, with only 6.3% overhead relative to FL training time.
  arXiv  Detail & Related papers  (2026-05-07T17:21:02Z)
• FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning [1.1393603788068776]
  Federated learning (FL) often degrades when clients hold heterogeneous non-Independent and Identically Distributed (non-IID) data.<n>This paper proposes FedEMA-Distill, a server-side procedure that combines an exponential moving average (EMA) of the global model with ensemble knowledge distillation from client-uploaded prediction logits.
  arXiv  Detail & Related papers  (2026-02-15T01:40:45Z)
• FedFeat+: A Robust Federated Learning Framework Through Federated Aggregation and Differentially Private Feature-Based Classifier Retraining [26.546920912051522]
  We propose the FedFeat+ framework, which distinctively separates feature extraction from classification.<n>We establish a balance between enhancing model accuracy and safeguarding individual privacy through the implementation of differential privacy mechanisms.<n>We validate our approach through empirical evaluations conducted on benchmark datasets, including CIFAR-10, CIFAR-100, MNIST, and FMNIST.
  arXiv  Detail & Related papers  (2025-04-08T13:12:38Z)
• Parametric Feature Transfer: One-shot Federated Learning with Foundation Models [14.97955440815159]
  In one-shot federated learning, clients collaboratively train a global model in a single round of communication. This paper introduces FedPFT, a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL.
  arXiv  Detail & Related papers  (2024-02-02T19:34:46Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• FedCliP: Federated Learning with Client Pruning [3.796320380104124]
  Federated learning (FL) is a newly emerging distributed learning paradigm. One fundamental bottleneck in FL is the heavy communication overheads between the distributed clients and the central server. We propose FedCliP, the first communication efficient FL training framework from a macro perspective.
  arXiv  Detail & Related papers  (2023-01-17T09:15:37Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• FL Games: A Federated Learning Framework for Distribution Shifts [71.98708418753786]
  Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. We propose FL GAMES, a game-theoretic framework for federated learning that learns causal features that are invariant across clients.
  arXiv  Detail & Related papers  (2022-10-31T22:59:03Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Unifying Distillation with Personalization in Federated Learning [1.8262547855491458]
  Federated learning (FL) is a decentralized privacy-preserving learning technique in which clients learn a joint collaborative model through a central aggregator without sharing their data. In this setting, all clients learn a single common predictor (FedAvg), which does not generalize well on each client's local data due to the statistical data heterogeneity among clients. In this paper, we address this problem with PersFL, a two-stage personalized learning algorithm. In the first stage, PersFL finds the optimal teacher model of each client during the FL training phase. In the second stage, PersFL distills the useful knowledge from
  arXiv  Detail & Related papers  (2021-05-31T17:54:29Z)
• Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
  Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
  arXiv  Detail & Related papers  (2021-04-27T18:23:54Z)
• CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning [2.2172881631608456]
  We introduce a new family of Federated Learning algorithms called CatFedAvg. It improves the communication efficiency but improves the quality of learning using a category coverage inNIST strategy. Our experiments show that an increase of 10% absolute points accuracy using the M dataset with 70% absolute points lower network transfer over FedAvg.
  arXiv  Detail & Related papers  (2020-11-14T06:52:02Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.