Federated Prompt Learning for Weather Foundation Models on Devices

• URL: http://arxiv.org/abs/2305.14244v2
• Date: Mon, 22 Apr 2024 00:50:38 GMT
• Title: Federated Prompt Learning for Weather Foundation Models on Devices
• Authors: Shengchao Chen, Guodong Long, Tao Shen, Jing Jiang, Chengqi Zhang,
• Abstract summary: On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing. This paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD) FedPoD enables devices to obtain highly customized models while maintaining communication efficiency.
• Score: 37.88417074427373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing, holds significance for supporting human activates. Federated Learning is a promising solution for such forecasting by enabling collaborative model training without sharing raw data. However, it faces three main challenges that hinder its reliability: (1) data heterogeneity among devices due to geographic differences; (2) data homogeneity within individual devices and (3) communication overload from sending large model parameters for collaboration. To address these challenges, this paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD), which enables devices to obtain highly customized models while maintaining communication efficiency. Concretely, our Adaptive Prompt Tuning leverages lightweight prompts guide frozen foundation model to generate more precise predictions, also conducts prompt-based multi-level communication to encourage multi-source knowledge fusion and regulate optimization. Additionally, Dynamic Graph Modeling constructs graphs from prompts, prioritizing collaborative training among devices with similar data distributions to against heterogeneity. Extensive experiments demonstrates FedPoD leads the performance among state-of-the-art baselines across various setting in real-world on-device weather forecasting datasets.

Related papers

• MITA: Bridging the Gap between Model and Data for Test-time Adaptation [68.62509948690698]
  Test-Time Adaptation (TTA) has emerged as a promising paradigm for enhancing the generalizability of models. We propose Meet-In-The-Middle based MITA, which introduces energy-based optimization to encourage mutual adaptation of the model and data from opposing directions.
  arXiv  Detail & Related papers  (2024-10-12T07:02:33Z)
• Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models [36.229082478423585]
  LM-Weather is a generic approach to taming pre-trained language models (PLMs) We introduce a lightweight personalized adapter into PLMs and endow it with weather pattern awareness. Experiments show LM-Weather outperforms the state-of-the-art results by a large margin across various tasks.
  arXiv  Detail & Related papers  (2024-05-24T15:25:09Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Personalized Federated Learning with Contextual Modulation and Meta-Learning [2.7716102039510564]
  Federated learning has emerged as a promising approach for training machine learning models on decentralized data sources. We propose a novel framework that combines federated learning with meta-learning techniques to enhance both efficiency and generalization capabilities.
  arXiv  Detail & Related papers  (2023-12-23T08:18:22Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Secure short-term load forecasting for smart grids with transformer-based federated learning [0.0]
  Electricity load forecasting is an essential task within smart grids to assist demand and supply balance. Fine-grained load profiles can expose users' electricity consumption behaviors, which raises privacy and security concerns. This paper presents a novel transformer-based deep learning approach with federated learning for short-term electricity load prediction.
  arXiv  Detail & Related papers  (2023-10-26T15:27:55Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Data [37.549578998407675]
  To tackle the global climate challenge, it urgently needs to develop a collaborative platform for comprehensive weather forecasting on large-scale meteorological data. This paper develops a foundation model across regions of understanding complex meteorological data and providing weather forecasting. A novel prompt learning mechanism has been adopted to satisfy low-resourced sensors' communication and computational constraints.
  arXiv  Detail & Related papers  (2023-01-22T16:47:05Z)
• Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks [50.68446003616802]
  Federated learning (FedL) has emerged as a popular technique for distributing model training over a set of wireless devices. We develop parallel successive learning (PSL), which expands the FedL architecture along three dimensions. Our analysis sheds light on the notion of cold vs. warmed up models, and model inertia in distributed machine learning.
  arXiv  Detail & Related papers  (2022-02-07T05:11:01Z)
• A Personalized Federated Learning Algorithm: an Application in Anomaly Detection [0.6700873164609007]
  Federated Learning (FL) has recently emerged as a promising method to overcome data privacy and transmission issues. In FL, datasets collected from different devices or sensors are used to train local models (clients) each of which shares its learning with a centralized model (server) This paper proposes a novel Personalized FedAvg (PC-FedAvg) which aims to control weights communication and aggregation augmented with a tailored learning algorithm to personalize the resulting models at each client.
  arXiv  Detail & Related papers  (2021-11-04T04:57:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.