Learning Temporal Consistency for Source-Free Video Domain Adaptation

• URL: http://arxiv.org/abs/2203.04559v1
• Date: Wed, 9 Mar 2022 07:33:36 GMT
• Title: Learning Temporal Consistency for Source-Free Video Domain Adaptation
• Authors: Yuecong Xu, Jianfei Yang, Haozhi Cao, Keyu Wu, Wu Min, Zhenghua Chen
• Abstract summary: In real-world applications, subjects and scenes in the source video domain should be irrelevant to those in the target video domain. To cope with such concern, a more practical domain adaptation scenario is formulated as the Source-Free Video-based Domain Adaptation (SFVDA) We propose a novel Attentive Temporal Consistent Network (ATCoN) to address SFVDA by learning temporal consistency.
• Score: 16.230405375192262
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video-based Unsupervised Domain Adaptation (VUDA) methods improve the robustness of video models, enabling them to be applied to action recognition tasks across different environments. However, these methods require constant access to source data during the adaptation process. Yet in many real-world applications, subjects and scenes in the source video domain should be irrelevant to those in the target video domain. With the increasing emphasis on data privacy, such methods that require source data access would raise serious privacy issues. Therefore, to cope with such concern, a more practical domain adaptation scenario is formulated as the Source-Free Video-based Domain Adaptation (SFVDA). Though there are a few methods for Source-Free Domain Adaptation (SFDA) on image data, these methods yield degenerating performance in SFVDA due to the multi-modality nature of videos, with the existence of additional temporal features. In this paper, we propose a novel Attentive Temporal Consistent Network (ATCoN) to address SFVDA by learning temporal consistency, guaranteed by two novel consistency objectives, namely feature consistency and source prediction consistency, performed across local temporal features. ATCoN further constructs effective overall temporal features by attending to local temporal features based on prediction confidence. Empirical results demonstrate the state-of-the-art performance of ATCoN across various cross-domain action recognition benchmarks.

Related papers

• Evidentially Calibrated Source-Free Time-Series Domain Adaptation with Temporal Imputation [38.88779207555418]
  Source-free domain adaptation (SFDA) aims to adapt a model pre-trained on a labeled source domain to an unlabeled target domain without access to source data. This paper proposes MAsk And imPUte (MAPU), a novel and effective approach for time series SFDA. We also introduce E-MAPU, which incorporates evidential uncertainty estimation to address the overconfidence issue inherent in softmax predictions.
  arXiv  Detail & Related papers  (2024-06-04T05:36:29Z)
• Source-Free Domain Adaptation with Temporal Imputation for Time Series Data [19.616201184995532]
  Source-free domain adaptation (SFDA) aims to adapt a pretrained model from a labeled source domain to an unlabeled target domain without access to the source domain data. Despite its prevalence in visual applications, SFDA is largely unexplored in time series applications. This paper presents a simple yet effective approach for source-free domain adaptation on time series data, namely MAsk and imPUte.
  arXiv  Detail & Related papers  (2023-07-14T14:22:03Z)
• CoSDA: Continual Source-Free Domain Adaptation [78.47274343972904]
  Without access to the source data, source-free domain adaptation (SFDA) transfers knowledge from a source-domain trained model to target domains. Recently, SFDA has gained popularity due to the need to protect the data privacy of the source domain, but it suffers from catastrophic forgetting on the source domain due to the lack of data. We propose a continual source-free domain adaptation approach named CoSDA, which employs a dual-speed optimized teacher-student model pair and is equipped with consistency learning capability.
  arXiv  Detail & Related papers  (2023-04-13T15:53:23Z)
• Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation [117.39092621796753]
  Source Domain Adaptation (SFDA) setup aims to adapt a source-trained model to the target domain without accessing source data. A novel method that takes full advantage of correlations oftemporal-information to tackle the absence of source data is proposed. Experiments show that PixelL achieves un-of-the-art performance on benchmarks compared to current UDA and SFDA approaches.
  arXiv  Detail & Related papers  (2023-03-25T05:06:23Z)
• Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective [37.45565756522847]
  We consider the generation of cross-domain videos from two sets of latent factors. TranSVAE framework is then developed to model such generation. Experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE.
  arXiv  Detail & Related papers  (2022-08-15T17:59:31Z)
• EXTERN: Leveraging Endo-Temporal Regularization for Black-box Video Domain Adaptation [36.8236874357225]
  Black-box Video Domain Adaptation (BVDA) is a more realistic yet challenging scenario where the source video model is provided only as a black-box predictor. We propose a novel Endo and eXo-TEmporal Regularized Network (EXTERN) by applying mask-to-mix strategies and video-tailored regularizations.
  arXiv  Detail & Related papers  (2022-08-10T07:09:57Z)
• Unsupervised Domain Adaptation for Video Transformers in Action Recognition [76.31442702219461]
  We propose a simple and novel UDA approach for video action recognition. Our approach builds a robust source model that better generalises to target domain. We report results on two video action benchmarks recognition for UDA.
  arXiv  Detail & Related papers  (2022-07-26T12:17:39Z)
• Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection [79.89082006155135]
  Unsupervised Domain Adaptation (UDA) is an effective approach to tackle the issue of domain shift. UDA methods try to align the source and target representations to improve the generalization on the target domain. The Source-Free Adaptation Domain (SFDA) setting aims to alleviate these concerns by adapting a source-trained model for the target domain without requiring access to the source data.
  arXiv  Detail & Related papers  (2022-03-29T17:50:43Z)
• Learning Cross-modal Contrastive Features for Video Domain Adaptation [138.75196499580804]
  We propose a unified framework for video domain adaptation, which simultaneously regularizes cross-modal and cross-domain feature representations. Specifically, we treat each modality in a domain as a view and leverage the contrastive learning technique with properly designed sampling strategies.
  arXiv  Detail & Related papers  (2021-08-26T18:14:18Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.