Wasserstein Transfer Learning

• URL: http://arxiv.org/abs/2505.17404v2
• Date: Thu, 23 Oct 2025 02:14:43 GMT
• Title: Wasserstein Transfer Learning
• Authors: Kaicheng Zhang, Sinian Zhang, Doudou Zhou, Yidong Zhou,
• Abstract summary: We introduce a novel transfer learning framework for regression models whose outputs are probability distributions residing in the Wasserstein space.<n>We propose an estimator with provable convergence rates, quantifying the impact of domain similarity on transfer efficiency.<n>For cases where the informative subset is unknown, we develop a data-driven transfer learning procedure designed to mitigate negative transfer.
• Score: 6.602088845993411
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer learning is a powerful paradigm for leveraging knowledge from source domains to enhance learning in a target domain. However, traditional transfer learning approaches often focus on scalar or multivariate data within Euclidean spaces, limiting their applicability to complex data structures such as probability distributions. To address this limitation, we introduce a novel transfer learning framework for regression models whose outputs are probability distributions residing in the Wasserstein space. When the informative subset of transferable source domains is known, we propose an estimator with provable asymptotic convergence rates, quantifying the impact of domain similarity on transfer efficiency. For cases where the informative subset is unknown, we develop a data-driven transfer learning procedure designed to mitigate negative transfer. The proposed methods are supported by rigorous theoretical analysis and are validated through extensive simulations and real-world applications. The code is available at https://github.com/h7nian/WaTL

Related papers

• Minimax optimal adaptive structured transfer learning through semi-parametric domain-varying coefficient model [9.091986429838117]
  We study a multi-source, single-target transfer learning problem under conditional distributional drift.<n>We develop an adaptive transfer learning estimator that selectively borrows strength from informative source domains.
  arXiv  Detail & Related papers  (2026-02-20T03:53:06Z)
• Transfer Learning Through Conditional Quantile Matching [3.86972243789112]
  We introduce a transfer learning framework for regression that leverages heterogeneous source domains to improve predictive performance in a data-scarce target domain.<n>Our approach learns a conditional generative model separately for each source domain and calibrates the generated responses to the target domain via conditional quantile matching.
  arXiv  Detail & Related papers  (2026-02-02T17:19:55Z)
• Quantifying Dataset Similarity to Guide Transfer Learning [1.6328866317851185]
  Cross-Learning Score ( CLS) measures dataset similarity through bidirectional performance between domains.<n> CLS can reliably predict whether transfer will improve or degrade performance.<n> CLS is efficient and fast to compute as it bypasses the problem of expensive distribution estimation for high-dimensional problems.
  arXiv  Detail & Related papers  (2025-10-13T00:18:35Z)
• Transfer Learning Under High-Dimensional Network Convolutional Regression Model [20.18595334666282]
  We propose a high-dimensional transfer learning framework based on network convolutional regression ( NCR)<n>Our methodology includes a two-step transfer learning algorithm that addresses domain shift between source and target networks.<n> Empirical evaluations, including simulations and a real-world application using Sina Weibo data, demonstrate substantial improvements in prediction accuracy.
  arXiv  Detail & Related papers  (2025-04-28T16:52:28Z)
• Transfer Learning through Enhanced Sufficient Representation: Enriching Source Domain Knowledge with Target Data [2.308168896770315]
  We introduce a novel method for transfer learning called Transfer learning through Enhanced Sufficient Representation (TESR)<n>Our approach begins by estimating a sufficient and invariant representation from the source domains.<n>This representation is then enhanced with an independent component derived from the target data, ensuring that it is sufficient for the target domain and adaptable to its specific characteristics.
  arXiv  Detail & Related papers  (2025-02-22T13:18:28Z)
• Deep Transfer $Q$-Learning for Offline Non-Stationary Reinforcement Learning [3.2839905453386162]
  This paper pioneers the study of transfer learning for dynamic decision scenarios modeled by non-stationary finite-horizon Markov decision processes.<n>We introduce a novel re-weighted targeting procedure'' to construct transferable RL samples'' and propose transfer deep $Q*$-learning''<n>Our analytical techniques for transfer learning in neural network approximation and transition density transfers have broader implications.
  arXiv  Detail & Related papers  (2025-01-08T23:03:18Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.<n>Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• A Recent Survey of Heterogeneous Transfer Learning [15.830786437956144]
  heterogeneous transfer learning has become a vital strategy in various tasks. We offer an extensive review of over 60 HTL methods, covering both data-based and model-based approaches. We explore applications in natural language processing, computer vision, multimodal learning, and biomedicine.
  arXiv  Detail & Related papers  (2023-10-12T16:19:58Z)
• Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfer [65.42096702428347]
  Graph Neural Networks (GNNs) aggregate information from neighboring nodes. Knowledge Bridge Learning (KBL) learns a knowledge-enhanced posterior distribution for target domains. Bridged-GNN includes an Adaptive Knowledge Retrieval module to build Bridged-Graph and a Graph Knowledge Transfer module.
  arXiv  Detail & Related papers  (2023-08-18T12:14:51Z)
• Estimation and inference for transfer learning with high-dimensional quantile regression [3.4510296013600374]
  We propose a transfer learning procedure in the framework of high-dimensional quantile regression models. We establish error bounds of transfer learning estimator based on delicately selected transferable source domains. By adopting data-splitting technique, we advocate a transferability detection approach that guarantees to circumvent negative transfer.
  arXiv  Detail & Related papers  (2022-11-26T14:40:19Z)
• Transfer learning with affine model transformation [18.13383101189326]
  This paper presents a general class of transfer learning regression called affine model transfer. It is shown that the affine model transfer broadly encompasses various existing methods, including the most common procedure based on neural feature extractors.
  arXiv  Detail & Related papers  (2022-10-18T10:50:24Z)
• Transfer Learning on Heterogeneous Feature Spaces for Treatment Effects Estimation [103.55894890759376]
  This paper introduces several building blocks that use representation learning to handle the heterogeneous feature spaces. We show how these building blocks can be used to recover transfer learning equivalents of the standard CATE learners.
  arXiv  Detail & Related papers  (2022-10-08T16:41:02Z)
• On Hypothesis Transfer Learning of Functional Linear Models [8.557392136621894]
  We study the transfer learning (TL) for the functional linear regression (FLR) under the Reproducing Kernel Space (RKHS) framework. We measure the similarity across tasks using RKHS distance, allowing the type of information being transferred tied to the properties of the imposed RKHS. Two algorithms are proposed: one conducts the transfer when positive sources are known, while the other leverages aggregation to achieve robust transfer without prior information about the sources.
  arXiv  Detail & Related papers  (2022-06-09T04:50:16Z)
• On Generalizing Beyond Domains in Cross-Domain Continual Learning [91.56748415975683]
  Deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.
  arXiv  Detail & Related papers  (2022-03-08T09:57:48Z)
• A Variational Bayesian Approach to Learning Latent Variables for Acoustic Knowledge Transfer [55.20627066525205]
  We propose a variational Bayesian (VB) approach to learning distributions of latent variables in deep neural network (DNN) models. Our proposed VB approach can obtain good improvements on target devices, and consistently outperforms 13 state-of-the-art knowledge transfer algorithms.
  arXiv  Detail & Related papers  (2021-10-16T15:54:01Z)
• Quantifying and Improving Transferability in Domain Generalization [53.16289325326505]
  Out-of-distribution generalization is one of the key challenges when transferring a model from the lab to the real world. We formally define transferability that one can quantify and compute in domain generalization. We propose a new algorithm for learning transferable features and test it over various benchmark datasets.
  arXiv  Detail & Related papers  (2021-06-07T14:04:32Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• What is being transferred in transfer learning? [51.6991244438545]
  We show that when training from pre-trained weights, the model stays in the same basin in the loss landscape. We present that when training from pre-trained weights, the model stays in the same basin in the loss landscape and different instances of such model are similar in feature space and close in parameter space.
  arXiv  Detail & Related papers  (2020-08-26T17:23:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.