Post-Training Dialogue Summarization using Pseudo-Paraphrasing
- URL: http://arxiv.org/abs/2204.13498v1
- Date: Thu, 28 Apr 2022 13:42:19 GMT
- Title: Post-Training Dialogue Summarization using Pseudo-Paraphrasing
- Authors: Qi Jia, Yizhu Liu, Haifeng Tang, Kenny Q. Zhu
- Abstract summary: We propose to post-train pretrained language models (PLMs) to rephrase from dialogue to narratives.
Comprehensive experiments show that our approach significantly improves vanilla PLMs on dialogue summarization.
- Score: 12.083992819138716
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Previous dialogue summarization techniques adapt large language models
pretrained on the narrative text by injecting dialogue-specific features into
the models. These features either require additional knowledge to recognize or
make the resulting models harder to tune. To bridge the format gap between
dialogues and narrative summaries in dialogue summarization tasks, we propose
to post-train pretrained language models (PLMs) to rephrase from dialogue to
narratives. After that, the model is fine-tuned for dialogue summarization as
usual. Comprehensive experiments show that our approach significantly improves
vanilla PLMs on dialogue summarization and outperforms other SOTA models by the
summary quality and implementation costs.
Related papers
- SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization [48.284512017469524]
Multi-turn dialogues are characterized by their extended length and the presence of turn-taking conversations.
Traditional language models often overlook the distinct features of these dialogues by treating them as regular text.
We propose a speaker-enhanced pre-training method for long dialogue summarization.
arXiv Detail & Related papers (2024-01-31T04:50:00Z) - FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for
Task-Oriented Dialogue [20.79359173822053]
We propose a novel dialogue pre-training model, FutureTOD, which distills future knowledge to the representation of the previous dialogue context.
Our intuition is that a good dialogue representation both learns local context information and predicts future information.
arXiv Detail & Related papers (2023-06-17T10:40:07Z) - Pre-training Multi-party Dialogue Models with Latent Discourse Inference [85.9683181507206]
We pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying.
To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model.
arXiv Detail & Related papers (2023-05-24T14:06:27Z) - Stabilized In-Context Learning with Pre-trained Language Models for Few
Shot Dialogue State Tracking [57.92608483099916]
Large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks.
For more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial.
We introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query.
arXiv Detail & Related papers (2023-02-12T15:05:10Z) - STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension [42.57581945778631]
Abstractive dialogue summarization has long been viewed as an important standalone task in natural language processing.
We propose a novel type of dialogue summarization task - STRUctured DiaLoguE Summarization.
We show that our STRUDEL dialogue comprehension model can significantly improve the dialogue comprehension performance of transformer encoder language models.
arXiv Detail & Related papers (2022-12-24T04:39:54Z) - DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain.
Our experiments show that DIONYSUS outperforms existing methods on six datasets.
arXiv Detail & Related papers (2022-12-20T06:21:21Z) - He Said, She Said: Style Transfer for Shifting the Perspective of
Dialogues [75.58367095888914]
We define a new style transfer task: perspective shift, which reframes a dialogue from informal first person to a formal third person rephrasing of the text.
As a sample application, we demonstrate that applying perspective shifting to a dialogue summarization dataset (SAMSum) substantially improves the zero-shot performance of extractive news summarization models.
arXiv Detail & Related papers (2022-10-27T14:16:07Z) - Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue
Comprehension [48.483910831143724]
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances.
We develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input.
arXiv Detail & Related papers (2022-03-19T05:20:25Z) - Response Generation with Context-Aware Prompt Learning [19.340498579331555]
We present a novel approach for pre-trained dialogue modeling that casts the dialogue generation problem as a prompt-learning task.
Instead of fine-tuning on limited dialogue data, our approach, DialogPrompt, learns continuous prompt embeddings optimized for dialogue contexts.
Our approach significantly outperforms the fine-tuning baseline and the generic prompt-learning methods.
arXiv Detail & Related papers (2021-11-04T05:40:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.