Style Control for Schema-Guided Natural Language Generation

• URL: http://arxiv.org/abs/2109.12211v1
• Date: Fri, 24 Sep 2021 21:47:58 GMT
• Title: Style Control for Schema-Guided Natural Language Generation
• Authors: Alicia Y. Tsai, Shereen Oraby, Vittorio Perera, Jiun-Yu Kao, Yuheng Du, Anjali Narayan-Chen, Tagyoung Chung, Dilek Hakkani-Tur
• Abstract summary: Natural Language Generation (NLG) for task-oriented dialogue systems focuses on communicating content accurately, fluently, and coherently. We focus on stylistic control and evaluation for schema-guided NLG, with joint goals of achieving both semantic and stylistic control.
• Score: 10.821250408348655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural Language Generation (NLG) for task-oriented dialogue systems focuses on communicating specific content accurately, fluently, and coherently. While these attributes are crucial for a successful dialogue, it is also desirable to simultaneously accomplish specific stylistic goals, such as response length, point-of-view, descriptiveness, sentiment, formality, and empathy. In this work, we focus on stylistic control and evaluation for schema-guided NLG, with joint goals of achieving both semantic and stylistic control. We experiment in detail with various controlled generation methods for large pretrained language models: specifically, conditional training, guided fine-tuning, and guided decoding. We discuss their advantages and limitations, and evaluate them with a broad range of automatic and human evaluation metrics. Our results show that while high style accuracy and semantic correctness are easier to achieve for more lexically-defined styles with conditional training, stylistic control is also achievable for more semantically complex styles using discriminator-based guided decoding methods. The results also suggest that methods that are more scalable (with less hyper-parameters tuning) and that disentangle content generation and stylistic variations are more effective at achieving semantic correctness and style accuracy.

Related papers

• SCAR: Efficient Instruction-Tuning for Large Language Models via Style Consistency-Aware Response Ranking [56.93151679231602]
  This research identifies two key stylistic elements in responses: linguistic form and semantic surprisal. Inspired by this, we introduce Style Consistency-Aware Response Ranking (SCAR) SCAR prioritizes instruction-response pairs in the training set based on their response stylistic consistency.
  arXiv  Detail & Related papers  (2024-06-16T10:10:37Z)
• StyleChat: Learning Recitation-Augmented Memory in LLMs for Stylized Dialogue Generation [43.29667566560533]
  We introduce a stylized dialogue dataset StyleEval with 38 styles by leveraging the generative power of Large Language Models (LLMs) We propose the stylized dialogue framework StyleChat via recitation-augmented memory strategy and multi-task style learning strategy to promote generalization ability.
  arXiv  Detail & Related papers  (2024-03-18T03:26:18Z)
• Dynamic Multi-Reward Weighting for Multi-Style Controllable Generation [15.959784404955402]
  Textual style expresses a diverse set of information, including interpersonal dynamics (e.g., formality) and the author's emotions or attitudes (e.g., disgust) An open question is how language models can be explicitly controlled so that they weave together target styles when generating text. One approach to such controlled generation is multi-objective reinforcement learning (RL) We investigate various formulations of multi-style rewards, including calibrated outputs from discriminators and dynamic weighting by discriminator magnitudes.
  arXiv  Detail & Related papers  (2024-02-21T22:02:37Z)
• Learning to Generate Text in Arbitrary Writing Styles [6.7308816341849695]
  It is desirable for language models to produce text in an author-specific style on the basis of a potentially small writing sample. We propose to guide a language model to generate text in a target style using contrastively-trained representations that capture stylometric features.
  arXiv  Detail & Related papers  (2023-12-28T18:58:52Z)
• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• Conversation Style Transfer using Few-Shot Learning [56.43383396058639]
  In this paper, we introduce conversation style transfer as a few-shot learning problem. We propose a novel in-context learning approach to solve the task with style-free dialogues as a pivot. We show that conversation style transfer can also benefit downstream tasks.
  arXiv  Detail & Related papers  (2023-02-16T15:27:00Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Context-Aware Language Modeling for Goal-Oriented Dialogue Systems [84.65707332816353]
  We formulate goal-oriented dialogue as a partially observed Markov decision process. We derive a simple and effective method to finetune language models in a goal-aware way. We evaluate our method on a practical flight-booking task using AirDialogue.
  arXiv  Detail & Related papers  (2022-04-18T17:23:11Z)
• Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
  We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
  arXiv  Detail & Related papers  (2020-10-11T08:27:07Z)
• StyleDGPT: Stylized Response Generation with Pre-trained Language Models [39.526613595499356]
  We introduce a KL loss and a style classifier to steer response generation towards the target style in both a word-level and a sentence-level. Our model can significantly outperform state-of-the-art methods in terms of both style consistency and contextual coherence.
  arXiv  Detail & Related papers  (2020-10-06T09:29:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.