A Closer Look at Smoothness in Domain Adversarial Training
- URL: http://arxiv.org/abs/2206.08213v1
- Date: Thu, 16 Jun 2022 14:31:38 GMT
- Title: A Closer Look at Smoothness in Domain Adversarial Training
- Authors: Harsh Rangwani, Sumukh K Aithal, Mayank Mishra, Arihant Jain, R.
Venkatesh Babu
- Abstract summary: We analyze the effect of smoothness enhancing formulations on domain adversarial training.
We find that converging to a smooth minima with respect to (w.r.t.) task loss stabilizes the adversarial training leading to better performance on target domain.
In contrast to task loss, our analysis shows that converging to smooth minima w.r.t. adversarial loss leads to sub-optimal generalization on the target domain.
- Score: 37.205372217498656
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Domain adversarial training has been ubiquitous for achieving invariant
representations and is used widely for various domain adaptation tasks. In
recent times, methods converging to smooth optima have shown improved
generalization for supervised learning tasks like classification. In this work,
we analyze the effect of smoothness enhancing formulations on domain
adversarial training, the objective of which is a combination of task loss (eg.
classification, regression, etc.) and adversarial terms. We find that
converging to a smooth minima with respect to (w.r.t.) task loss stabilizes the
adversarial training leading to better performance on target domain. In
contrast to task loss, our analysis shows that converging to smooth minima
w.r.t. adversarial loss leads to sub-optimal generalization on the target
domain. Based on the analysis, we introduce the Smooth Domain Adversarial
Training (SDAT) procedure, which effectively enhances the performance of
existing domain adversarial methods for both classification and object
detection tasks. Our analysis also provides insight into the extensive usage of
SGD over Adam in the community for domain adversarial training.
Related papers
- Improve Cross-domain Mixed Sampling with Guidance Training for Adaptive Segmentation [9.875170018805768]
Unsupervised Domain Adaptation (UDA) endeavors to adjust models trained on a source domain to perform well on a target domain without requiring additional annotations.
We propose a novel auxiliary task called Guidance Training.
This task facilitates the effective utilization of cross-domain mixed sampling techniques while mitigating distribution shifts from the real world.
We demonstrate the efficacy of our approach by integrating it with existing methods, consistently improving performance.
arXiv Detail & Related papers (2024-03-22T07:12:48Z) - AIR-DA: Adversarial Image Reconstruction for Unsupervised Domain
Adaptive Object Detection [28.22783703278792]
Adrial Image Reconstruction (AIR) as the regularizer to facilitate the adversarial training of the feature extractor.
Our evaluations across several datasets of challenging domain shifts demonstrate that the proposed method outperforms all previous methods.
arXiv Detail & Related papers (2023-03-27T16:51:51Z) - Domain Adaptation with Adversarial Training on Penultimate Activations [82.9977759320565]
Enhancing model prediction confidence on unlabeled target data is an important objective in Unsupervised Domain Adaptation (UDA)
We show that this strategy is more efficient and better correlated with the objective of boosting prediction confidence than adversarial training on input images or intermediate features.
arXiv Detail & Related papers (2022-08-26T19:50:46Z) - Enhancing Adversarial Training with Feature Separability [52.39305978984573]
We introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to boost the intra-class feature similarity and increase inter-class feature variance.
Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.
arXiv Detail & Related papers (2022-05-02T04:04:23Z) - Domain Adversarial Training: A Game Perspective [80.3821370633883]
This paper defines optimal solutions in domain-adversarial training from a game theoretical perspective.
We show that descent in domain-adversarial training can violate the convergence guarantees of the gradient, oftentimes hindering the transfer performance.
Ours are easy to implement, free of additional parameters, and can be plugged into any domain-adversarial framework.
arXiv Detail & Related papers (2022-02-10T22:17:30Z) - Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive
Learning [62.7588467386166]
We leverage contrastive learning to bridge the domain gap by aligning the features of structurally similar label patches across domains.
Our approach consistently outperforms state-of-the-art unsupervised and semi-supervised methods on two challenging domain adaptive segmentation tasks.
arXiv Detail & Related papers (2021-04-22T13:39:12Z) - MetaAlign: Coordinating Domain Alignment and Classification for
Unsupervised Domain Adaptation [84.90801699807426]
This paper proposes an effective meta-optimization based strategy dubbed MetaAlign.
We treat the domain alignment objective and the classification objective as the meta-train and meta-test tasks in a meta-learning scheme.
Experimental results demonstrate the effectiveness of our proposed method on top of various alignment-based baseline approaches.
arXiv Detail & Related papers (2021-03-25T03:16:05Z) - Learning a Domain-Agnostic Visual Representation for Autonomous Driving
via Contrastive Loss [25.798361683744684]
Domain-Agnostic Contrastive Learning (DACL) is a two-stage unsupervised domain adaptation framework with cyclic adversarial training and contrastive loss.
Our proposed approach achieves better performance in the monocular depth estimation task compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-03-10T07:06:03Z) - Unsupervised Domain Adaptive Object Detection using Forward-Backward
Cyclic Adaptation [13.163271874039191]
We present a novel approach to perform the unsupervised domain adaptation for object detection through forward-backward cyclic (FBC) training.
Recent adversarial training based domain adaptation methods have shown their effectiveness on minimizing domain discrepancy via marginal feature distributions alignment.
We propose Forward-Backward Cyclic Adaptation, which iteratively computes adaptation from source to target via backward hopping and from target to source via forward passing.
arXiv Detail & Related papers (2020-02-03T06:24:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.