Inference Time Style Control for Summarization
- URL: http://arxiv.org/abs/2104.01724v1
- Date: Mon, 5 Apr 2021 00:27:18 GMT
- Title: Inference Time Style Control for Summarization
- Authors: Shuyang Cao and Lu Wang
- Abstract summary: We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model.
In experiments of summarizing with simplicity control, automatic evaluation and human judges both find our models producing outputs in simpler languages while still informative.
- Score: 6.017006996402699
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: How to generate summaries of different styles without requiring corpora in
the target styles, or training separate models? We present two novel methods
that can be deployed during summary decoding on any pre-trained
Transformer-based summarization model. (1) Decoder state adjustment instantly
modifies decoder final states with externally trained style scorers, to
iteratively refine the output against a target style. (2) Word unit prediction
constrains the word usage to impose strong lexical control during generation.
In experiments of summarizing with simplicity control, automatic evaluation and
human judges both find our models producing outputs in simpler languages while
still informative. We also generate news headlines with various ideological
leanings, which can be distinguished by humans with a reasonable probability.
Related papers
- Unsupervised Text Style Transfer with Deep Generative Models [12.801169425020225]
We present a general framework for unsupervised text style transfer with deep generative models.
Our framework is able to unify previous embedding and prototype methods as two special forms.
It also provides a principled perspective to explain previously proposed techniques in the field such as aligned encoder and adversarial training.
arXiv Detail & Related papers (2023-08-31T09:29:35Z) - Attributable and Scalable Opinion Summarization [79.87892048285819]
We generate abstractive summaries by decoding frequent encodings, and extractive summaries by selecting the sentences assigned to the same frequent encodings.
Our method is attributable, because the model identifies sentences used to generate the summary as part of the summarization process.
It scales easily to many hundreds of input reviews, because aggregation is performed in the latent space rather than over long sequences of tokens.
arXiv Detail & Related papers (2023-05-19T11:30:37Z) - Collocation2Text: Controllable Text Generation from Guide Phrases in
Russian [0.0]
Collocation2Text is a plug-and-play method for automatic controllable text generation in Russian.
The method is based on two interacting models: the autoregressive language ruGPT-3 model and the autoencoding language ruRoBERTa model.
Experiments on generating news articles using the proposed method showed its effectiveness for automatically generated fluent texts.
arXiv Detail & Related papers (2022-06-18T17:10:08Z) - Twist Decoding: Diverse Generators Guide Each Other [116.20780037268801]
We introduce Twist decoding, a simple and general inference algorithm that generates text while benefiting from diverse models.
Our method does not assume the vocabulary, tokenization or even generation order is shared.
arXiv Detail & Related papers (2022-05-19T01:27:53Z) - Word-Level Style Control for Expressive, Non-attentive Speech Synthesis [1.8262960053058506]
It attempts to learn word-level stylistic and prosodic representations of the speech data, with the aid of two encoders.
We find that the resulting model gives both word-level and global control over the style, as well as prosody transfer capabilities.
arXiv Detail & Related papers (2021-11-19T12:03:53Z) - Style Equalization: Unsupervised Learning of Controllable Generative
Sequence Models [23.649790871960644]
We tackle the training-inference mismatch encountered during unsupervised learning of controllable generative sequence models.
By introducing a style transformation module that we call style equalization, we enable training using different content and style samples.
Our models achieve state-of-the-art style replication with a similar mean style opinion score as the real data.
arXiv Detail & Related papers (2021-10-06T16:17:57Z) - Exemplar-Controllable Paraphrasing and Translation using Bitext [57.92051459102902]
We adapt models from prior work to be able to learn solely from bilingual text (bitext)
Our single proposed model can perform four tasks: controlled paraphrase generation in both languages and controlled machine translation in both language directions.
arXiv Detail & Related papers (2020-10-12T17:02:50Z) - Neural Syntactic Preordering for Controlled Paraphrase Generation [57.5316011554622]
Our work uses syntactic transformations to softly "reorder'' the source sentence and guide our neural paraphrasing model.
First, given an input sentence, we derive a set of feasible syntactic rearrangements using an encoder-decoder model.
Next, we use each proposed rearrangement to produce a sequence of position embeddings, which encourages our final encoder-decoder paraphrase model to attend to the source words in a particular order.
arXiv Detail & Related papers (2020-05-05T09:02:25Z) - Few-Shot Learning for Opinion Summarization [117.70510762845338]
Opinion summarization is the automatic creation of text reflecting subjective information expressed in multiple documents.
In this work, we show that even a handful of summaries is sufficient to bootstrap generation of the summary text.
Our approach substantially outperforms previous extractive and abstractive methods in automatic and human evaluation.
arXiv Detail & Related papers (2020-04-30T15:37:38Z) - Exploring Explainable Selection to Control Abstractive Summarization [51.74889133688111]
We develop a novel framework that focuses on explainability.
A novel pair-wise matrix captures the sentence interactions, centrality, and attribute scores.
A sentence-deployed attention mechanism in the abstractor ensures the final summary emphasizes the desired content.
arXiv Detail & Related papers (2020-04-24T14:39:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.