Gradient-guided Unsupervised Text Style Transfer via Contrastive Learning

• URL: http://arxiv.org/abs/2202.00469v1
• Date: Sun, 23 Jan 2022 12:45:00 GMT
• Title: Gradient-guided Unsupervised Text Style Transfer via Contrastive Learning
• Authors: Chenghao Fan, Ziao Li, Wei wei
• Abstract summary: We propose a gradient-guided model through a contrastive paradigm for text style transfer, to explicitly gather similar semantic sentences. Experiments on two datasets show the effectiveness of our proposed approach, as compared to the state-of-the-arts.
• Score: 6.799826701166569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text style transfer is a challenging text generation problem, which aims at altering the style of a given sentence to a target one while keeping its content unchanged. Since there is a natural scarcity of parallel datasets, recent works mainly focus on solving the problem in an unsupervised manner. However, previous gradient-based works generally suffer from the deficiencies as follows, namely: (1) Content migration. Previous approaches lack explicit modeling of content invariance and are thus susceptible to content shift between the original sentence and the transferred one. (2) Style misclassification. A natural drawback of the gradient-guided approaches is that the inference process is homogeneous with a line of adversarial attack, making latent optimization easily becomes an attack to the classifier due to misclassification. This leads to difficulties in achieving high transfer accuracy. To address the problems, we propose a novel gradient-guided model through a contrastive paradigm for text style transfer, to explicitly gather similar semantic sentences, and to design a siamese-structure based style classifier for alleviating such two issues, respectively. Experiments on two datasets show the effectiveness of our proposed approach, as compared to the state-of-the-arts.

Related papers

• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer [30.622772801446132]
  We propose a semi-supervised framework for text style transfer. First, the learning process is bootstrapped with supervision guided by automatically constructed pseudo-parallel pairs. Then the model learns from unlabeled data via reinforcement rewards.
  arXiv  Detail & Related papers  (2022-05-19T05:18:06Z)
• Transductive Learning for Unsupervised Text Style Transfer [60.65782243927698]
  Unsupervised style transfer models are mainly based on an inductive learning approach. We propose a novel transductive learning approach based on a retrieval-based context-aware style representation.
  arXiv  Detail & Related papers  (2021-09-16T08:57:20Z)
• A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations [27.129551973093008]
  This paper introduces a novel variational upper bound to the mutual information between an attribute and the latent code of an encoder. It aims at controlling the approximation error via the Renyi's divergence, leading to both better disentangled representations and a precise control of the desirable degree of disentanglement. We show the superiority of this method on fair classification and on textual style transfer tasks.
  arXiv  Detail & Related papers  (2021-05-06T14:05:06Z)
• StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer [90.6768813620898]
  Style transfer aims to controllably generate text with targeted stylistic changes while maintaining core meaning from the source sentence constant. We introduce a large-scale benchmark, StylePTB, with paired sentences undergoing 21 fine-grained stylistic changes spanning atomic lexical, syntactic, semantic, and thematic transfers of text. We find that existing methods on StylePTB struggle to model fine-grained changes and have an even more difficult time composing multiple styles.
  arXiv  Detail & Related papers  (2021-04-12T04:25:09Z)
• GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer [119.70961704127157]
  Non-parallel text style transfer has attracted increasing research interests in recent years. Current approaches still lack the ability to preserve the content and even logic of original sentences. We propose a method called Graph Transformer based Auto-GTAE, which models a sentence as a linguistic graph and performs feature extraction and style transfer at the graph level.
  arXiv  Detail & Related papers  (2021-02-01T11:08:45Z)
• Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer [60.07283363509065]
  Unsupervised style transfer aims to change the style of an input sentence while preserving its original content. We propose a novel attentional sequence-to-sequence model that exploits the relevance of each output word to the target style. Experimental results show that our proposed model achieves state-of-the-art performance in terms of both transfer accuracy and content preservation.
  arXiv  Detail & Related papers  (2020-05-05T10:24:28Z)
• Contextual Text Style Transfer [73.66285813595616]
  Contextual Text Style Transfer aims to translate a sentence into a desired style with its surrounding context taken into account. We propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context. Two new benchmarks, Enron-Context and Reddit-Context, are introduced for formality and offensiveness style transfer.
  arXiv  Detail & Related papers  (2020-04-30T23:01:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.