Tweet to News Conversion: An Investigation into Unsupervised
Controllable Text Generation
- URL: http://arxiv.org/abs/2008.09333v1
- Date: Fri, 21 Aug 2020 06:56:57 GMT
- Title: Tweet to News Conversion: An Investigation into Unsupervised
Controllable Text Generation
- Authors: Zishan Ahmad, Mukuntha N S, Asif Ekbal, Pushpak Bhattacharyya
- Abstract summary: In this paper, we define the task of constructing a coherent paragraph from a set of disaster domain tweets.
We tackle the problem by building two systems in pipeline. The first system focuses on unsupervised style transfer and converts the individual tweets into news sentences.
The second system stitches together the outputs from the first system to form a coherent news paragraph.
- Score: 46.74654716230366
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Text generator systems have become extremely popular with the advent of
recent deep learning models such as encoder-decoder. Controlling the
information and style of the generated output without supervision is an
important and challenging Natural Language Processing (NLP) task. In this
paper, we define the task of constructing a coherent paragraph from a set of
disaster domain tweets, without any parallel data. We tackle the problem by
building two systems in pipeline. The first system focuses on unsupervised
style transfer and converts the individual tweets into news sentences. The
second system stitches together the outputs from the first system to form a
coherent news paragraph. We also propose a novel training mechanism, by
splitting the sentences into propositions and training the second system to
merge the sentences. We create a validation and test set consisting of
tweet-sets and their equivalent news paragraphs to perform empirical
evaluation. In a completely unsupervised setting, our model was able to achieve
a BLEU score of 19.32, while successfully transferring styles and joining
tweets to form a meaningful news paragraph.
Related papers
- Unsupervised Text Style Transfer via LLMs and Attention Masking with
Multi-way Interactions [18.64326057581588]
Unsupervised Text Style Transfer (UTST) has emerged as a critical task within the domain of Natural Language Processing (NLP)
We propose four ways of interactions, that are pipeline framework with tuned orders; knowledge distillation from Large Language Models (LLMs) to attention masking model; in-context learning with constructed parallel examples.
We empirically show these multi-way interactions can improve the baselines in certain perspective of style strength, content preservation and text fluency.
arXiv Detail & Related papers (2024-02-21T09:28:02Z) - Improving Cascaded Unsupervised Speech Translation with Denoising
Back-translation [70.33052952571884]
We propose to build a cascaded speech translation system without leveraging any kind of paired data.
We use fully unpaired data to train our unsupervised systems and evaluate our results on CoVoST 2 and CVSS.
arXiv Detail & Related papers (2023-05-12T13:07:51Z) - R2D2: Robust Data-to-Text with Replacement Detection [16.53137103104244]
We introduce R2D2, a training framework that addresses unfaithful Data-to-Text generation.
We argue that the poor entity retrieval capability of D2T systems is one of the primary sources of unfaithfulness.
Our experimental results show that R2D2 systems could effectively mitigate the unfaithful text generation.
arXiv Detail & Related papers (2022-05-25T03:29:25Z) - Using BERT Encoding and Sentence-Level Language Model for Sentence
Ordering [0.9134244356393667]
We propose an algorithm for sentence ordering in a corpus of short stories.
Our proposed method uses a language model based on Universal Transformers (UT) that captures sentences' dependencies by employing an attention mechanism.
The proposed model includes three components: Sentence, Language Model, and Sentence Arrangement with Brute Force Search.
arXiv Detail & Related papers (2021-08-24T23:03:36Z) - AGGGEN: Ordering and Aggregating while Generating [12.845842212733695]
We present AGGGEN, a data-to-text model which re-introduces two explicit sentence planning stages into neural data-to-text systems.
AGGGEN performs sentence planning at the same time as generating text by learning latent alignments between input representation and target text.
arXiv Detail & Related papers (2021-06-10T08:14:59Z) - Conditioned Text Generation with Transfer for Closed-Domain Dialogue
Systems [65.48663492703557]
We show how to optimally train and control the generation of intent-specific sentences using a conditional variational autoencoder.
We introduce a new protocol called query transfer that allows to leverage a large unlabelled dataset.
arXiv Detail & Related papers (2020-11-03T14:06:10Z) - Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking.
We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
arXiv Detail & Related papers (2020-10-24T11:55:28Z) - Unsupervised Extractive Summarization by Pre-training Hierarchical
Transformers [107.12125265675483]
Unsupervised extractive document summarization aims to select important sentences from a document without using labeled summaries during training.
Existing methods are mostly graph-based with sentences as nodes and edge weights measured by sentence similarities.
We find that transformer attentions can be used to rank sentences for unsupervised extractive summarization.
arXiv Detail & Related papers (2020-10-16T08:44:09Z) - Unsupervised Bitext Mining and Translation via Self-trained Contextual
Embeddings [51.47607125262885]
We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text.
We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training.
We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods.
arXiv Detail & Related papers (2020-10-15T14:04:03Z) - Stacked DeBERT: All Attention in Incomplete Data for Text Classification [8.900866276512364]
We propose Stacked DeBERT, short for Stacked Denoising Bidirectional Representations from Transformers.
Our model shows improved F1-scores and better robustness in informal/incorrect texts present in tweets and in texts with Speech-to-Text error in sentiment and intent classification tasks.
arXiv Detail & Related papers (2020-01-01T04:49:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.