Image Sentiment Transfer

• URL: http://arxiv.org/abs/2006.11337v1
• Date: Fri, 19 Jun 2020 19:28:08 GMT
• Title: Image Sentiment Transfer
• Authors: Tianlang Chen, Wei Xiong, Haitian Zheng, Jiebo Luo
• Abstract summary: We introduce an important but still unexplored research task -- image sentiment transfer. We propose an effective and flexible framework that performs image sentiment transfer at the object level. For the core object-level sentiment transfer, we propose a novel Sentiment-aware GAN (SentiGAN)
• Score: 84.91653085312277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we introduce an important but still unexplored research task -- image sentiment transfer. Compared with other related tasks that have been well-studied, such as image-to-image translation and image style transfer, transferring the sentiment of an image is more challenging. Given an input image, the rule to transfer the sentiment of each contained object can be completely different, making existing approaches that perform global image transfer by a single reference image inadequate to achieve satisfactory performance. In this paper, we propose an effective and flexible framework that performs image sentiment transfer at the object level. It first detects the objects and extracts their pixel-level masks, and then performs object-level sentiment transfer guided by multiple reference images for the corresponding objects. For the core object-level sentiment transfer, we propose a novel Sentiment-aware GAN (SentiGAN). Both global image-level and local object-level supervisions are imposed to train SentiGAN. More importantly, an effective content disentanglement loss cooperating with a content alignment step is applied to better disentangle the residual sentiment-related information of the input image. Extensive quantitative and qualitative experiments are performed on the object-oriented VSO dataset we create, demonstrating the effectiveness of the proposed framework.

Related papers

• Transformer based Multitask Learning for Image Captioning and Object Detection [13.340784876489927]
  This work introduces a novel multitask learning framework that combines image captioning and object detection into a joint model. We propose TICOD, Transformer-based Image Captioning and Object detection model for jointly training both tasks. Our model outperforms the baselines from image captioning literature by achieving a 3.65% improvement in BERTScore.
  arXiv  Detail & Related papers  (2024-03-10T19:31:13Z)
• Situational Perception Guided Image Matting [16.1897179939677]
  We propose a Situational Perception Guided Image Matting (SPG-IM) method that mitigates subjective bias of matting annotations. SPG-IM can better associate inter-objects and object-to-environment saliency, and compensate the subjective nature of image matting.
  arXiv  Detail & Related papers  (2022-04-20T07:35:51Z)
• Unsupervised Image-to-Image Translation with Generative Prior [103.54337984566877]
  Unsupervised image-to-image translation aims to learn the translation between two visual domains without paired data. We present a novel framework, Generative Prior-guided UN Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
  arXiv  Detail & Related papers  (2022-04-07T17:59:23Z)
• Panoptic-based Object Style-Align for Image-to-Image Translation [2.226472061870956]
  We propose panoptic-based object style-align generative adversarial networks (POSA-GANs) for image-to-image translation. The proposed method was systematically compared with different competing methods and obtained significant improvement on both image quality and object recognition performance for translated images.
  arXiv  Detail & Related papers  (2021-12-03T14:28:11Z)
• Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types [50.1843146606122]
  A simple form of transfer learning is common in current state-of-the-art computer vision models. Previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains.
  arXiv  Detail & Related papers  (2021-03-24T16:24:20Z)
• Global Image Sentiment Transfer [90.26415735432576]
  The proposed framework consists of a reference image retrieval step and a global sentiment transfer step. The retrieved reference images are more content-related against the algorithm based on the perceptual loss. The proposed sentiment transfer algorithm can transfer the sentiment of images while ensuring the content structure of the input image intact.
  arXiv  Detail & Related papers  (2020-06-22T03:22:25Z)
• Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies [72.67344725725961]
  We present a novel unsupervised learning approach to image landmark discovery by incorporating the inter-subject landmark consistencies on facial images. This is achieved via an inter-subject mapping module that transforms original subject landmarks based on an auxiliary subject-related structure. To recover from the transformed images back to the original subject, the landmark detector is forced to learn spatial locations that contain the consistent semantic meanings both for the paired intra-subject images and between the paired inter-subject images.
  arXiv  Detail & Related papers  (2020-04-16T20:38:16Z)
• Cross-domain Correspondence Learning for Exemplar-based Image Translation [59.35767271091425]
  We present a framework for exemplar-based image translation, which synthesizes a photo-realistic image from the input in a distinct domain. The output has the style (e.g., color, texture) in consistency with the semantically corresponding objects in the exemplar. We show that our method is superior to state-of-the-art methods in terms of image quality significantly.
  arXiv  Detail & Related papers  (2020-04-12T09:10:57Z)
• Learning Transformation-Aware Embeddings for Image Forensics [15.484408315588569]
  Image Provenance Analysis aims at discovering relationships among different manipulated image versions that share content. One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates. This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations.
  arXiv  Detail & Related papers  (2020-01-13T22:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.