GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer

• URL: http://arxiv.org/abs/2102.00769v1
• Date: Mon, 1 Feb 2021 11:08:45 GMT
• Title: GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer
• Authors: Yukai Shi, Sen Zhang, Chenxing Zhou, Xiaodan Liang, Xiaojun Yang, Liang Lin
• Abstract summary: Non-parallel text style transfer has attracted increasing research interests in recent years. Current approaches still lack the ability to preserve the content and even logic of original sentences. We propose a method called Graph Transformer based Auto-GTAE, which models a sentence as a linguistic graph and performs feature extraction and style transfer at the graph level.
• Score: 119.70961704127157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Non-parallel text style transfer has attracted increasing research interests in recent years. Despite successes in transferring the style based on the encoder-decoder framework, current approaches still lack the ability to preserve the content and even logic of original sentences, mainly due to the large unconstrained model space or too simplified assumptions on latent embedding space. Since language itself is an intelligent product of humans with certain grammars and has a limited rule-based model space by its nature, relieving this problem requires reconciling the model capacity of deep neural networks with the intrinsic model constraints from human linguistic rules. To this end, we propose a method called Graph Transformer based Auto Encoder (GTAE), which models a sentence as a linguistic graph and performs feature extraction and style transfer at the graph level, to maximally retain the content and the linguistic structure of original sentences. Quantitative experiment results on three non-parallel text style transfer tasks show that our model outperforms state-of-the-art methods in content preservation, while achieving comparable performance on transfer accuracy and sentence naturalness.

Related papers

• A Transformer-based Approach for Arabic Offline Handwritten Text Recognition [0.0]
  We introduce two alternative architectures for recognizing offline Arabic handwritten text. Our approach can model language dependencies and relies only on the attention mechanism, thereby making it more parallelizable and less complex. Our evaluation on the Arabic KHATT dataset demonstrates that our proposed method outperforms the current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-07-27T17:51:52Z)
• Pre-Training a Graph Recurrent Network for Language Representation [34.4554387894105]
  We consider a graph recurrent network for language model pre-training, which builds a graph structure for each sequence with local token-level communications. We find that our model can generate more diverse outputs with less contextualized feature redundancy than existing attention-based models.
  arXiv  Detail & Related papers  (2022-09-08T14:12:15Z)
• StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing [73.81778485157234]
  Long texts usually involve more complicated author linguistic preferences such as discourse structures than sentences. We formulate the task of non-parallel story author-style transfer, which requires transferring an input story into a specified author style. We use an additional training objective to disentangle stylistic features from the learned discourse representation to prevent the model from degenerating to an auto-encoder.
  arXiv  Detail & Related papers  (2022-08-29T08:47:49Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer [60.07283363509065]
  Unsupervised style transfer aims to change the style of an input sentence while preserving its original content. We propose a novel attentional sequence-to-sequence model that exploits the relevance of each output word to the target style. Experimental results show that our proposed model achieves state-of-the-art performance in terms of both transfer accuracy and content preservation.
  arXiv  Detail & Related papers  (2020-05-05T10:24:28Z)
• Contextual Text Style Transfer [73.66285813595616]
  Contextual Text Style Transfer aims to translate a sentence into a desired style with its surrounding context taken into account. We propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context. Two new benchmarks, Enron-Context and Reddit-Context, are introduced for formality and offensiveness style transfer.
  arXiv  Detail & Related papers  (2020-04-30T23:01:12Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.