Heterogeneous Federated Learning Systems for Time-Series Power Consumption Prediction with Multi-Head Embedding Mechanism

• URL: http://arxiv.org/abs/2501.12136v1
• Date: Tue, 21 Jan 2025 13:49:38 GMT
• Title: Heterogeneous Federated Learning Systems for Time-Series Power Consumption Prediction with Multi-Head Embedding Mechanism
• Authors: Jia-Hao Syu, Jerry Chun-Wei Lin, Gautam Srivastava, Unil Yun,
• Abstract summary: Time-series prediction is increasingly popular in a variety of applications, such as smart factories and smart transportation. Existing models lack discussion of collaborative learning and privacy issues among multiple clients. We propose Multi-Head Heterogeneous Federated Learning (MHHFL) systems that consist of multiple head networks.
• Score: 45.85115919116268
• License:
• Abstract: Time-series prediction is increasingly popular in a variety of applications, such as smart factories and smart transportation. Researchers have used various techniques to predict power consumption, but existing models lack discussion of collaborative learning and privacy issues among multiple clients. To address these issues, we propose Multi-Head Heterogeneous Federated Learning (MHHFL) systems that consist of multiple head networks, which independently act as carriers for federated learning. In the federated period, each head network is embedded into 2-dimensional vectors and shared with the centralized source pool. MHHFL then selects appropriate source networks and blends the head networks as knowledge transfer in federated learning. The experimental results show that the proposed MHHFL systems significantly outperform the benchmark and state-of-the-art systems and reduce the prediction error by 24.9% to 94.1%. The ablation studies demonstrate the effectiveness of the proposed mechanisms in the MHHFL (head network embedding and selection mechanisms), which significantly outperforms traditional federated average and random transfer.

Related papers

• Distributed Multi-Head Learning Systems for Power Consumption Prediction [59.293903039988884]
  We propose Distributed Multi-Head learning (DMH) systems for power consumption prediction in smart factories. DMH systems are designed as distributed and split learning, reducing the client-to-server transmission cost. DMH-E system reduces the error of the state-of-the-art systems by 14.5% to 24.0%.
  arXiv  Detail & Related papers  (2025-01-21T13:46:23Z)
• Heterogeneous Federated Learning System for Sparse Healthcare Time-Series Prediction [26.197000069634207]
  We design dense and sparse feature tensors to deal with sparsity of data sources. Heterogeneous federated learning is developed to share asynchronous parts of networks and select appropriate models for knowledge transfer.
  arXiv  Detail & Related papers  (2025-01-21T13:38:06Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Deep Equilibrium Models Meet Federated Learning [71.57324258813675]
  This study explores the problem of Federated Learning (FL) by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning.
  arXiv  Detail & Related papers  (2023-05-29T22:51:40Z)
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence [52.83633954857744]
  We investigate real-time machine learning in a federated edge intelligence (FEI) system. FEI systems exhibit heterogenous communication and computational resource distribution. We propose a time-sensitive federated learning (TS-FL) framework to minimize the overall run-time for collaboratively training a shared ML model.
  arXiv  Detail & Related papers  (2023-01-26T08:13:22Z)
• GowFed -- A novel Federated Network Intrusion Detection System [0.15469452301122172]
  This work presents GowFed, a novel network threat detection system that combines the usage of Gower Dissimilarity matrices and Federated averaging. Different approaches of GowFed have been developed based on state-of the-art knowledge: (1) a vanilla version; and (2) a version instrumented with an attention mechanism. Overall, GowFed intends to be the first stepping stone towards the combined usage of Federated Learning and Gower Dissimilarity matrices to detect network threats in industrial-level networks.
  arXiv  Detail & Related papers  (2022-10-28T23:53:37Z)
• Communication-Efficient Hierarchical Federated Learning for IoT Heterogeneous Systems with Imbalanced Data [42.26599494940002]
  Federated learning (FL) is a distributed learning methodology that allows multiple nodes to cooperatively train a deep learning model. This paper studies the potential of hierarchical FL in IoT heterogeneous systems. It proposes an optimized solution for user assignment and resource allocation on multiple edge nodes.
  arXiv  Detail & Related papers  (2021-07-14T08:32:39Z)
• A Transductive Multi-Head Model for Cross-Domain Few-Shot Learning [72.30054522048553]
  We present a new method, Transductive Multi-Head Few-Shot learning (TMHFS), to address the Cross-Domain Few-Shot Learning challenge. The proposed methods greatly outperform the strong baseline, fine-tuning, on four different target domains.
  arXiv  Detail & Related papers  (2020-06-08T02:39:59Z)
• Ternary Compression for Communication-Efficient Federated Learning [17.97683428517896]
  Federated learning provides a potential solution to privacy-preserving and secure machine learning. We propose a ternary federated averaging protocol (T-FedAvg) to reduce the upstream and downstream communication of federated learning systems. Our results show that the proposed T-FedAvg is effective in reducing communication costs and can even achieve slightly better performance on non-IID data.
  arXiv  Detail & Related papers  (2020-03-07T11:55:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.