Few-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages

• URL: http://arxiv.org/abs/2110.07385v1
• Date: Thu, 14 Oct 2021 14:16:39 GMT
• Title: Few-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages
• Authors: Kalpesh Krishna, Deepak Nathani, Xavier Garcia, Bidisha Samanta, Partha Talukdar
• Abstract summary: Style transfer is the task of rewriting an input sentence into a target style while preserving its content. We push the state-of-the-art for few-shot style transfer with a new method modeling the stylistic difference between paraphrases. Our model achieves 2-3x better performance and output diversity in formality transfer and code-mixing addition across five Indian languages.
• Score: 13.980482277351523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Style transfer is the task of rewriting an input sentence into a target style while approximately preserving its content. While most prior literature assumes access to large style-labelled corpora, recent work (Riley et al. 2021) has attempted "few-shot" style transfer using only 3-10 sentences at inference for extracting the target style. In this work we consider one such low resource setting where no datasets are available: style transfer for Indian languages. We find that existing few-shot methods perform this task poorly, with a strong tendency to copy inputs verbatim. We push the state-of-the-art for few-shot style transfer with a new method modeling the stylistic difference between paraphrases. When compared to prior work using automatic and human evaluations, our model achieves 2-3x better performance and output diversity in formality transfer and code-mixing addition across five Indian languages. Moreover, our method is better able to control the amount of style transfer using an input scalar knob. We report promising qualitative results for several attribute transfer directions, including sentiment transfer, text simplification, gender neutralization and text anonymization, all without retraining the model. Finally we found model evaluation to be difficult due to the lack of evaluation datasets and metrics for Indian languages. To facilitate further research in formality transfer for Indic languages, we crowdsource annotations for 4000 sentence pairs in four languages, and use this dataset to design our automatic evaluation suite.

Related papers

• Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment [50.27950279695363]
  The transfer performance is often hindered when a low-resource target language is written in a different script than the high-resource source language. Inspired by recent work that uses transliteration to address this problem, our paper proposes a transliteration-based post-pretraining alignment (PPA) method.
  arXiv  Detail & Related papers  (2024-06-28T08:59:24Z)
• Are Large Language Models Actually Good at Text Style Transfer? [0.17751300245073598]
  We analyze the performance of large language models (LLMs) on Text Style Transfer (TST) TST involves modifying the linguistic style of a text while preserving its core content. We evaluate the capabilities of pre-trained LLMs using zero-shot and few-shot prompting as well as parameter-efficient finetuning on publicly available datasets.
  arXiv  Detail & Related papers  (2024-06-09T18:45:41Z)
• Don't lose the message while paraphrasing: A study on content preserving style transfer [61.38460184163704]
  Content preservation is critical for real-world applications of style transfer studies. We compare various style transfer models on the example of the formality transfer domain. We conduct a precise comparative study of several state-of-the-art techniques for style transfer.
  arXiv  Detail & Related papers  (2023-08-17T15:41:08Z)
• Conversation Style Transfer using Few-Shot Learning [56.43383396058639]
  In this paper, we introduce conversation style transfer as a few-shot learning problem. We propose a novel in-context learning approach to solve the task with style-free dialogues as a pivot. We show that conversation style transfer can also benefit downstream tasks.
  arXiv  Detail & Related papers  (2023-02-16T15:27:00Z)
• StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing [73.81778485157234]
  Long texts usually involve more complicated author linguistic preferences such as discourse structures than sentences. We formulate the task of non-parallel story author-style transfer, which requires transferring an input story into a specified author style. We use an additional training objective to disentangle stylistic features from the learned discourse representation to prevent the model from degenerating to an auto-encoder.
  arXiv  Detail & Related papers  (2022-08-29T08:47:49Z)
• Reformulating Unsupervised Style Transfer as Paraphrase Generation [48.83148014000888]
  We reformulate unsupervised style transfer as a paraphrase generation problem. We present a simple methodology based on fine-tuning pretrained language models on automatically generated paraphrase data. We also pivot to a more real-world style transfer setting by collecting a large dataset of 15M sentences in 11 diverse styles.
  arXiv  Detail & Related papers  (2020-10-12T13:31:01Z)
• PGST: a Polyglot Gender Style Transfer method [5.220940151628734]
  P GST is a novel polyglot text style transfer approach in the gender domain. It is feasible to apply a style transfer method in multiple languages by fulfilling our method's predefined elements. To demonstrate our method's multilingual applicability, we applied our method on both English and Persian corpora.
  arXiv  Detail & Related papers  (2020-09-02T13:15:11Z)
• Politeness Transfer: A Tag and Generate Approach [167.9924201435888]
  This paper introduces a new task of politeness transfer. It involves converting non-polite sentences to polite sentences while preserving the meaning. We design a tag and generate pipeline that identifies stylistic attributes and subsequently generates a sentence in the target style.
  arXiv  Detail & Related papers  (2020-04-29T15:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.