Emotional Semantics-Preserved and Feature-Aligned CycleGAN for Visual Emotion Adaptation

• URL: http://arxiv.org/abs/2011.12470v1
• Date: Wed, 25 Nov 2020 01:31:01 GMT
• Title: Emotional Semantics-Preserved and Feature-Aligned CycleGAN for Visual Emotion Adaptation
• Authors: Sicheng Zhao, Xuanbai Chen, Xiangyu Yue, Chuang Lin, Pengfei Xu, Ravi Krishna, Jufeng Yang, Guiguang Ding, Alberto L. Sangiovanni-Vincentelli, Kurt Keutzer
• Abstract summary: Unsupervised domain adaptation (UDA) studies the problem of transferring models trained on one labeled source domain to another unlabeled target domain. In this paper, we focus on UDA in visual emotion analysis for both emotion distribution learning and dominant emotion classification. We propose a novel end-to-end cycle-consistent adversarial model, termed CycleEmotionGAN++.
• Score: 85.20533077846606
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Thanks to large-scale labeled training data, deep neural networks (DNNs) have obtained remarkable success in many vision and multimedia tasks. However, because of the presence of domain shift, the learned knowledge of the well-trained DNNs cannot be well generalized to new domains or datasets that have few labels. Unsupervised domain adaptation (UDA) studies the problem of transferring models trained on one labeled source domain to another unlabeled target domain. In this paper, we focus on UDA in visual emotion analysis for both emotion distribution learning and dominant emotion classification. Specifically, we design a novel end-to-end cycle-consistent adversarial model, termed CycleEmotionGAN++. First, we generate an adapted domain to align the source and target domains on the pixel-level by improving CycleGAN with a multi-scale structured cycle-consistency loss. During the image translation, we propose a dynamic emotional semantic consistency loss to preserve the emotion labels of the source images. Second, we train a transferable task classifier on the adapted domain with feature-level alignment between the adapted and target domains. We conduct extensive UDA experiments on the Flickr-LDL & Twitter-LDL datasets for distribution learning and ArtPhoto & FI datasets for emotion classification. The results demonstrate the significant improvements yielded by the proposed CycleEmotionGAN++ as compared to state-of-the-art UDA approaches.

Related papers

• SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing [14.007392647145448]
  UDA enables models to learn from unlabeled target domain data while training on labeled source domain data. We propose integrating contrastive learning into UDA, enhancing the model's capacity to capture semantic information. Our SimSeg method outperforms existing approaches, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-17T11:59:39Z)
• Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation [17.875516787157018]
  We study how to harness the knowledge priors learned by 2D visual foundation models to produce more accurate labels for unlabeled target domains. Our method is evaluated on various autonomous driving datasets and the results demonstrate a significant improvement for 3D segmentation task.
  arXiv  Detail & Related papers  (2024-03-15T03:58:17Z)
• Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation [210.46684938698485]
  We study unsupervised domain adaptation (UDA) for semantic segmentation. We find that, by decreasing/increasing the proportion of training samples from the target domain, the 'learning ability' is strengthened/weakened. We propose a novel dual teacher-student (DTS) framework and equip it with a bidirectional learning strategy.
  arXiv  Detail & Related papers  (2023-03-16T05:04:10Z)
• Target and Task specific Source-Free Domain Adaptive Image Segmentation [73.78898054277538]
  We propose a two-stage approach for source-free domain adaptive image segmentation. We focus on generating target-specific pseudo labels while suppressing high entropy regions. In the second stage, we focus on adapting the network for task-specific representation.
  arXiv  Detail & Related papers  (2022-03-29T17:50:22Z)
• Domain Adaptation on Semantic Segmentation for Aerial Images [3.946367634483361]
  We propose a novel unsupervised domain adaptation framework to address domain shift in semantic image segmentation. We also apply entropy minimization on the target domain to produce high-confident prediction. We show improvement over state-of-the-art methods in terms of various metrics.
  arXiv  Detail & Related papers  (2020-12-03T20:58:27Z)
• DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets [96.92018649136217]
  We present a novel Domain-oriented Feature Embedding (DoFE) framework to improve the generalization ability of CNNs on unseen target domains. Our DoFE framework dynamically enriches the image features with additional domain prior knowledge learned from multi-source domains. Our framework generates satisfying segmentation results on unseen datasets and surpasses other domain generalization and network regularization methods.
  arXiv  Detail & Related papers  (2020-10-13T07:28:39Z)
• Domain Adaptive Semantic Segmentation Using Weak Labels [115.16029641181669]
  We propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. We develop a weak-label classification module to enforce the network to attend to certain categories. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting.
  arXiv  Detail & Related papers  (2020-07-30T01:33:57Z)
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation [58.38749495295393]
  Domain adaptation aims to learn a transferable model to bridge the domain shift between one labeled source domain and another sparsely labeled or unlabeled target domain. Recent multi-source domain adaptation (MDA) methods do not consider the pixel-level alignment between sources and target. We propose a novel MDA framework to address these challenges.
  arXiv  Detail & Related papers  (2020-02-19T21:22:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.