Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective

• URL: http://arxiv.org/abs/2502.13573v1
• Date: Wed, 19 Feb 2025 09:27:03 GMT
• Title: Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective
• Authors: Yuan Yao, Xiaopu Zhang, Yu Zhang, Jian Jin, Qiang Yang,
• Abstract summary: Semi-supervised heterogeneous domain adaptation (SHDA) addresses learning across domains with distinct feature representations and distributions. We conduct extensive experiments on about 330 SHDA tasks, employing two supervised learning methods and seven representative SHDA methods. We find that the transferable knowledge in SHDA primarily stems from the transferability and discriminability of the source domain.
• Score: 16.414320476939306
• License:
• Abstract: Semi-supervised heterogeneous domain adaptation (SHDA) addresses learning across domains with distinct feature representations and distributions, where source samples are labeled while most target samples are unlabeled, with only a small fraction labeled. Moreover, there is no one-to-one correspondence between source and target samples. Although various SHDA methods have been developed to tackle this problem, the nature of the knowledge transferred across heterogeneous domains remains unclear. This paper delves into this question from an empirical perspective. We conduct extensive experiments on about 330 SHDA tasks, employing two supervised learning methods and seven representative SHDA methods. Surprisingly, our observations indicate that both the category and feature information of source samples do not significantly impact the performance of the target domain. Additionally, noise drawn from simple distributions, when used as source samples, may contain transferable knowledge. Based on this insight, we perform a series of experiments to uncover the underlying principles of transferable knowledge in SHDA. Specifically, we design a unified Knowledge Transfer Framework (KTF) for SHDA. Based on the KTF, we find that the transferable knowledge in SHDA primarily stems from the transferability and discriminability of the source domain. Consequently, ensuring those properties in source samples, regardless of their origin (e.g., image, text, noise), can enhance the effectiveness of knowledge transfer in SHDA tasks. The codes and datasets are available at https://github.com/yyyaoyuan/SHDA.

Related papers

• Semi Supervised Heterogeneous Domain Adaptation via Disentanglement and Pseudo-Labelling [4.33404822906643]
  Semi-supervised domain adaptation methods leverage information from a source labelled domain to generalize over a scarcely labelled target domain. Such a setting is denoted as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA) We introduce SHeDD (Semi-supervised Heterogeneous Domain Adaptation via Disentanglement) an end-to-end neural framework tailored to learning a target domain.
  arXiv  Detail & Related papers  (2024-06-20T08:02:49Z)
• Confidence-based Visual Dispersal for Few-shot Unsupervised Domain Adaptation [39.112032738643656]
  Unsupervised domain adaptation aims to transfer knowledge from a fully-labeled source domain to an unlabeled target domain. We propose a Confidence-based Visual Dispersal Transfer learning method (C-VisDiT) for FUDA. We conduct extensive experiments on Office-31, Office-Home, VisDA-C, and DomainNet benchmark datasets and the results demonstrate that the proposed C-VisDiT significantly outperforms state-of-the-art FUDA methods.
  arXiv  Detail & Related papers  (2023-09-27T11:16:51Z)
• Exploring Incompatible Knowledge Transfer in Few-shot Image Generation [107.81232567861117]
  Few-shot image generation learns to generate diverse and high-fidelity images from a target domain using a few reference samples. Existing F SIG methods select, preserve and transfer prior knowledge from a source generator to learn the target generator. We propose knowledge truncation, which is a complementary operation to knowledge preservation and is implemented by a lightweight pruning-based method.
  arXiv  Detail & Related papers  (2023-04-15T14:57:15Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift [82.14087963690561]
  Multi-source domain adaptation (MSDA) addresses the challenge of learning a label prediction function for an unlabeled target domain. We present an intricate causal generative model by introducing latent noises across domains, along with a latent content variable and a latent style variable. The proposed approach showcases exceptional performance and efficacy on both simulated and real-world datasets.
  arXiv  Detail & Related papers  (2022-08-30T11:25:15Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• UMAD: Universal Model Adaptation under Domain and Category Shift [138.12678159620248]
  Universal Model ADaptation (UMAD) framework handles both UDA scenarios without access to source data. We develop an informative consistency score to help distinguish unknown samples from known samples. Experiments on open-set and open-partial-set UDA scenarios demonstrate that UMAD exhibits comparable, if not superior, performance to state-of-the-art data-dependent methods.
  arXiv  Detail & Related papers  (2021-12-16T01:22:59Z)
• A Framework for Supervised Heterogeneous Transfer Learning using Dynamic Distribution Adaptation and Manifold Regularization [3.476077954140922]
  We present a framework called TLF that builds a classifier for the target domain having only few labeled training records. We handle distribution divergence by simultaneously optimizing the structural risk functional, joint distributions between domains, and the manifold consistency underlying marginal distributions. We evaluate TLF on seven publicly available natural datasets and compare the performance of TLF against the performance of eleven state-of-the-art techniques.
  arXiv  Detail & Related papers  (2021-08-27T14:00:09Z)
• Multi-source Domain Adaptation for Visual Sentiment Classification [92.53780541232773]
  We propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN) To handle data from multiple source domains, MSGAN learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.
  arXiv  Detail & Related papers  (2020-01-12T08:37:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.