STEER: Unified Style Transfer with Expert Reinforcement

• URL: http://arxiv.org/abs/2311.07167v1
• Date: Mon, 13 Nov 2023 09:02:30 GMT
• Title: STEER: Unified Style Transfer with Expert Reinforcement
• Authors: Skyler Hallinan, Faeze Brahman, Ximing Lu, Jaehun Jung, Sean Welleck, Yejin Choi
• Abstract summary: STEER: Unified Style Transfer with Expert Reinforcement, is a unified frame-work developed to overcome the challenge of limited parallel data for style transfer. We show STEER is robust, maintaining its style transfer capabilities on out-of-domain data, and surpassing nearly all baselines across various styles.
• Score: 71.3995732115262
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While text style transfer has many applications across natural language processing, the core premise of transferring from a single source style is unrealistic in a real-world setting. In this work, we focus on arbitrary style transfer: rewriting a text from an arbitrary, unknown style to a target style. We propose STEER: Unified Style Transfer with Expert Reinforcement, a unified frame-work developed to overcome the challenge of limited parallel data for style transfer. STEER involves automatically generating a corpus of style-transfer pairs using a product of experts during decoding. The generated offline data is then used to pre-train an initial policy before switching to online, off-policy reinforcement learning for further improvements via fine-grained reward signals. STEER is unified and can transfer to multiple target styles from an arbitrary, unknown source style, making it particularly flexible and efficient. Experimental results on a challenging dataset with text from a diverse set of styles demonstrate state-of-the-art results compared to competitive baselines. Remarkably, STEER outperforms the 175B parameter instruction-tuned GPT-3 on overall style transfer quality, despite being 226 times smaller in size. We also show STEER is robust, maintaining its style transfer capabilities on out-of-domain data, and surpassing nearly all baselines across various styles. The success of our method highlights the potential of RL algorithms when augmented with controllable decoding to overcome the challenge of limited data supervision.

Related papers

• SETTP: Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning [22.04285529067442]
  Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning is proposed. SETTP learns source style-level prompts containing fundamental style characteristics from high-resource style transfer. Experiments show SETTP requires only 1/20th of the data volume to achieve performance comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-22T11:34:48Z)
• Authorship Style Transfer with Policy Optimization [26.34892894935038]
  Authorship style transfer aims to rewrite a given text into a specified target while preserving the original meaning in the source. Existing approaches rely on the availability of a large number of target style exemplars for model training.
  arXiv  Detail & Related papers  (2024-03-12T19:34:54Z)
• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer [82.37710853235535]
  Unsupervised text style transfer task aims to rewrite a text into target style while preserving its main content. Traditional methods rely on the use of a fixed-sized vector to regulate text style, which is difficult to accurately convey the style strength for each individual token. Our proposed method addresses this issue by assigning individual style vector to each token in a text, allowing for fine-grained control and manipulation of the style strength.
  arXiv  Detail & Related papers  (2023-06-12T13:12:29Z)
• Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfer [83.1333306079676]
  In this paper, we devise a novel Transformer model termed as emphMaster specifically for style transfer. In the proposed model, different Transformer layers share a common group of parameters, which (1) reduces the total number of parameters, (2) leads to more robust training convergence, and (3) is readily to control the degree of stylization. Experiments demonstrate the superiority of Master under both zero-shot and few-shot style transfer settings.
  arXiv  Detail & Related papers  (2023-04-24T04:46:39Z)
• Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer [30.622772801446132]
  We propose a semi-supervised framework for text style transfer. First, the learning process is bootstrapped with supervision guided by automatically constructed pseudo-parallel pairs. Then the model learns from unlabeled data via reinforcement rewards.
  arXiv  Detail & Related papers  (2022-05-19T05:18:06Z)
• StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer [90.6768813620898]
  Style transfer aims to controllably generate text with targeted stylistic changes while maintaining core meaning from the source sentence constant. We introduce a large-scale benchmark, StylePTB, with paired sentences undergoing 21 fine-grained stylistic changes spanning atomic lexical, syntactic, semantic, and thematic transfers of text. We find that existing methods on StylePTB struggle to model fine-grained changes and have an even more difficult time composing multiple styles.
  arXiv  Detail & Related papers  (2021-04-12T04:25:09Z)
• Reformulating Unsupervised Style Transfer as Paraphrase Generation [48.83148014000888]
  We reformulate unsupervised style transfer as a paraphrase generation problem. We present a simple methodology based on fine-tuning pretrained language models on automatically generated paraphrase data. We also pivot to a more real-world style transfer setting by collecting a large dataset of 15M sentences in 11 diverse styles.
  arXiv  Detail & Related papers  (2020-10-12T13:31:01Z)
• Fair Transfer of Multiple Style Attributes in Text [26.964711594103566]
  We show that the transfer of multiple styles cannot be achieved by sequentially performing multiple single-style transfers. We propose a neural network architecture for fairly transferring multiple style attributes in a given text.
  arXiv  Detail & Related papers  (2020-01-18T15:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.