Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach

• URL: http://arxiv.org/abs/2604.06727v1
• Date: Wed, 08 Apr 2026 06:50:37 GMT
• Title: Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
• Abstract summary: Heterogeneity in time series data is more pronounced than in vision or language.<n>Existing efforts on training time series foundation models (TSFMs) from scratch are often trained with mixed-batch strategies.<n>We propose a fine-grained learning method that distills invariant knowledge from heterogeneous series.
• Score: 39.76134852465009
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heterogeneity in time series data is more pronounced than in vision or language, as temporal dynamics vary substantially across domains and tasks. Existing efforts on training time series foundation models (TSFMs) from scratch are often trained with mixed-batch strategies that merge large-scale datasets, which can cause gradient conflicts and degrade representation quality. To address this, we propose a fine-grained learning method that distills invariant knowledge from heterogeneous series while reducing cross-domain interference. We characterize heterogeneity at two levels: inter-domain and intra-domain. To tackle this bi-level heterogeneity, we design a federated learning method that mitigates intra-domain conflicts by enforcing domain-invariant and semantically consistent representations through local regularization, and addresses inter-domain discrepancies by enhancing cross-domain collaboration via domain-aware aggregation. Experiments across diverse benchmarks show that TSFMs trained with our method consistently outperform both centralized and federated TSFM baselines in point and probabilistic forecasting, while also achieving competitive zero-shot performance at scale, offering a flexible pathway for training TSFMs from scratch in heterogeneous environments.

Related papers

• Bridging Granularity Gaps: Hierarchical Semantic Learning for Cross-domain Few-shot Segmentation [10.442470344182993]
  Cross-domain Few-shot (CD-FSS) aims to segment novel classes from target domains that are not involved in training.<n>We propose a Hierarchical Semantic Learning framework to tackle this problem.
  arXiv  Detail & Related papers  (2025-11-15T13:13:26Z)
• Cross-Domain Conditional Diffusion Models for Time Series Imputation [34.400748070956006]
  Cross-domain time series imputation is an underexplored data-centric research task.<n>Existing approaches primarily focus on the single-domain setting.<n>Our proposed solution integrates shared spectral components from both domains while retaining domain-specific temporal structures.
  arXiv  Detail & Related papers  (2025-06-14T09:09:07Z)
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification [57.945437355714155]
  Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions.<n>Existing approaches focus on single-source domain generalization to unseen target domains.<n>We propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data.
  arXiv  Detail & Related papers  (2024-12-05T06:15:08Z)
• Learning with Alignments: Tackling the Inter- and Intra-domain Shifts for Cross-multidomain Facial Expression Recognition [16.864390181629044]
  We propose a novel Learning with Alignments CMFER framework, named LA-CMFER, to handle both inter- and intra-domain shifts. Based on this, LA-CMFER presents a dual-level inter-domain alignment method to force the model to prioritize hard-to-align samples in knowledge transfer. To address the intra-domain shifts, LA-CMFER introduces a multi-view intra-domain alignment method with a multi-view consistency constraint.
  arXiv  Detail & Related papers  (2024-07-08T07:43:06Z)
• Hypernetwork-Driven Model Fusion for Federated Domain Generalization [26.492360039272942]
  Federated Learning (FL) faces significant challenges with domain shifts in heterogeneous data. We propose a robust framework, coined as hypernetwork-based Federated Fusion (hFedF), using hypernetworks for non-linear aggregation. Our method employs client-specific embeddings and gradient alignment techniques to manage domain generalization effectively.
  arXiv  Detail & Related papers  (2024-02-10T15:42:03Z)
• Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning [60.058083574671834]
  This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation. For heterogeneous issue, we leverage irrelevant unlabeled public data for communication. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
  arXiv  Detail & Related papers  (2023-09-28T09:32:27Z)
• Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment [17.086123737443714]
  Anomaly segmentation plays a pivotal role in identifying atypical objects in images, crucial for hazard detection in autonomous driving systems. While existing methods demonstrate noteworthy results on synthetic data, they often fail to consider the disparity between synthetic and real-world data domains. We introduce the Multi-Granularity Cross-Domain Alignment framework, tailored to harmonize features across domains at both the scene and individual sample levels.
  arXiv  Detail & Related papers  (2023-08-16T22:54:49Z)
• IDA: Informed Domain Adaptive Semantic Segmentation [51.12107564372869]
  We propose an Domain Informed Adaptation (IDA) model, a self-training framework that mixes the data based on class-level segmentation performance. In our IDA model, the class-level performance is tracked by an expected confidence score (ECS) and we then use a dynamic schedule to determine the mixing ratio for data in different domains. Our proposed method is able to outperform the state-of-the-art UDA-SS method by a margin of 1.1 mIoU in the adaptation of GTA-V to Cityscapes and of 0.9 mIoU in the adaptation of SYNTHIA to City
  arXiv  Detail & Related papers  (2023-03-05T18:16:34Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation [74.3349233035632]
  Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) do not consider an inter-class variation within the target domain itself or estimated category. We introduce a learnable clustering module, and a novel domain adaptation framework called cross-domain grouping and alignment. Our method consistently boosts the adaptation performance in semantic segmentation, outperforming the state-of-the-arts on various domain adaptation settings.
  arXiv  Detail & Related papers  (2020-12-15T11:36:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.