Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
- URL: http://arxiv.org/abs/2010.09211v1
- Date: Mon, 19 Oct 2020 04:25:10 GMT
- Title: Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
- Authors: Nakul Agarwal, Yi-Ting Chen, Behzad Dariush, Ming-Hsuan Yang
- Abstract summary: S-temporal action localization is an important problem in computer vision.
We propose an end-to-end unsupervised domain adaptation algorithm.
We show that significant performance gain can be achieved when spatial and temporal features are adapted separately or jointly.
- Score: 69.12982544509427
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Spatio-temporal action localization is an important problem in computer
vision that involves detecting where and when activities occur, and therefore
requires modeling of both spatial and temporal features. This problem is
typically formulated in the context of supervised learning, where the learned
classifiers operate on the premise that both training and test data are sampled
from the same underlying distribution. However, this assumption does not hold
when there is a significant domain shift, leading to poor generalization
performance on the test data. To address this, we focus on the hard and novel
task of generalizing training models to test samples without access to any
labels from the latter for spatio-temporal action localization by proposing an
end-to-end unsupervised domain adaptation algorithm. We extend the
state-of-the-art object detection framework to localize and classify actions.
In order to minimize the domain shift, three domain adaptation modules at image
level (temporal and spatial) and instance level (temporal) are designed and
integrated. We design a new experimental setup and evaluate the proposed method
and different adaptation modules on the UCF-Sports, UCF-101 and JHMDB benchmark
datasets. We show that significant performance gain can be achieved when
spatial and temporal features are adapted separately, or jointly for the most
effective results.
Related papers
- DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment [7.768332621617199]
We introduce a strong DETR-based detector named Domain Adaptive detection TRansformer ( DATR) for unsupervised domain adaptation of object detection.
Our proposed DATR incorporates a mean-teacher based self-training framework, utilizing pseudo-labels generated by the teacher model to further mitigate domain bias.
Experiments demonstrate superior performance and generalization capabilities of our proposed DATR in multiple domain adaptation scenarios.
arXiv Detail & Related papers (2024-05-20T03:48:45Z) - STAT: Towards Generalizable Temporal Action Localization [56.634561073746056]
Weakly-supervised temporal action localization (WTAL) aims to recognize and localize action instances with only video-level labels.
Existing methods suffer from severe performance degradation when transferring to different distributions.
We propose GTAL, which focuses on improving the generalizability of action localization methods.
arXiv Detail & Related papers (2024-04-20T07:56:21Z) - One-Shot Domain Adaptive and Generalizable Semantic Segmentation with
Class-Aware Cross-Domain Transformers [96.51828911883456]
Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data.
Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation.
We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
arXiv Detail & Related papers (2022-12-14T15:54:15Z) - Exploiting Domain Transferability for Collaborative Inter-level Domain
Adaptive Object Detection [17.61278045720336]
Domain adaptation for object detection (DAOD) has recently drawn much attention owing to its capability of detecting target objects without any annotations.
Previous works focus on aligning features extracted from partial levels in a two-stage detector via adversarial training.
We introduce a novel framework for ProposalD with three proposed components: Multi-scale-aware Uncertainty Attention (MUA), Transferable Region Network (TRPN), and Dynamic Instance Sampling (DIS)
arXiv Detail & Related papers (2022-07-20T01:50:26Z) - Deep face recognition with clustering based domain adaptation [57.29464116557734]
We propose a new clustering-based domain adaptation method designed for face recognition task in which the source and target domain do not share any classes.
Our method effectively learns the discriminative target feature by aligning the feature domain globally, and, at the meantime, distinguishing the target clusters locally.
arXiv Detail & Related papers (2022-05-27T12:29:11Z) - Self-supervised Autoregressive Domain Adaptation for Time Series Data [9.75443057146649]
Unsupervised domain adaptation (UDA) has successfully addressed the domain shift problem for visual applications.
These approaches may have limited performance for time series data due to the following reasons.
We propose a Self-supervised Autoregressive Domain Adaptation (SLARDA) framework to address these limitations.
arXiv Detail & Related papers (2021-11-29T08:17:23Z) - Dynamic Feature Alignment for Semi-supervised Domain Adaptation [23.67093835143]
We propose to use dynamic feature alignment to address both inter- and intra-domain discrepancy.
Our approach, which doesn't require extensive tuning or adversarial training, significantly improves the state of the art for semi-supervised domain adaptation.
arXiv Detail & Related papers (2021-10-18T22:26:27Z) - AFAN: Augmented Feature Alignment Network for Cross-Domain Object
Detection [90.18752912204778]
Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications.
We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training.
Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
arXiv Detail & Related papers (2021-06-10T05:01:20Z) - Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.
In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts.
We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
arXiv Detail & Related papers (2020-07-06T17:59:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.