Adversarially Masked Video Consistency for Unsupervised Domain Adaptation

• URL: http://arxiv.org/abs/2403.16242v1
• Date: Sun, 24 Mar 2024 17:13:46 GMT
• Title: Adversarially Masked Video Consistency for Unsupervised Domain Adaptation
• Authors: Xiaoyu Zhu, Junwei Liang, Po-Yao Huang, Alex Hauptmann,
• Abstract summary: We study the problem of unsupervised domain adaptation for egocentric videos. We propose a transformer-based model to learn class-discriminative and domain-invariant feature representations.
• Score: 11.947273267877208
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We study the problem of unsupervised domain adaptation for egocentric videos. We propose a transformer-based model to learn class-discriminative and domain-invariant feature representations. It consists of two novel designs. The first module is called Generative Adversarial Domain Alignment Network with the aim of learning domain-invariant representations. It simultaneously learns a mask generator and a domain-invariant encoder in an adversarial way. The domain-invariant encoder is trained to minimize the distance between the source and target domain. The masking generator, conversely, aims at producing challenging masks by maximizing the domain distance. The second is a Masked Consistency Learning module to learn class-discriminative representations. It enforces the prediction consistency between the masked target videos and their full forms. To better evaluate the effectiveness of domain adaptation methods, we construct a more challenging benchmark for egocentric videos, U-Ego4D. Our method achieves state-of-the-art performance on the Epic-Kitchen and the proposed U-Ego4D benchmark.

Related papers

• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains [73.54897096088149]
  We propose a Domain-invariant Masked AutoEncoder (DiMAE) for self-supervised learning from multi-domains. The core idea is to augment the input image with style noise from different domains and then reconstruct the image from the embedding of the augmented image. Experiments on PACS and DomainNet illustrate that DiMAE achieves considerable gains compared with recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-10T09:49:40Z)
• Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing [55.73722120043086]
  We introduce Contrast and Mix (CoMix), a new contrastive learning framework that aims to learn discriminative invariant feature representations for unsupervised video domain adaptation. First, we utilize temporal contrastive learning to bridge the domain gap by maximizing the similarity between encoded representations of an unlabeled video at two different speeds. Second, we propose a novel extension to the temporal contrastive loss by using background mixing that allows additional positives per anchor, thus adapting contrastive learning to leverage action semantics shared across both domains.
  arXiv  Detail & Related papers  (2021-10-28T14:03:29Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Domain adaptation techniques for improved cross-domain study of galaxy mergers [0.0]
  In astronomy, neural networks are often trained on simulated data with the prospect of being applied to real observations. Here we demonstrate the use of two techniques - Maximum Mean Discrepancy (MMD) and adversarial training with Domain Adversarial Neural Networks (DANN) We show how the addition of either MMD or adversarial training greatly improves the performance of the classifier on the target domain when compared to conventional machine learning algorithms.
  arXiv  Detail & Related papers  (2020-11-06T20:42:32Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)
• Unsupervised Domain Adaptive Object Detection using Forward-Backward Cyclic Adaptation [13.163271874039191]
  We present a novel approach to perform the unsupervised domain adaptation for object detection through forward-backward cyclic (FBC) training. Recent adversarial training based domain adaptation methods have shown their effectiveness on minimizing domain discrepancy via marginal feature distributions alignment. We propose Forward-Backward Cyclic Adaptation, which iteratively computes adaptation from source to target via backward hopping and from target to source via forward passing.
  arXiv  Detail & Related papers  (2020-02-03T06:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.