THINK: A Novel Conversation Model for Generating Grammatically Correct and Coherent Responses

• URL: http://arxiv.org/abs/2105.13630v1
• Date: Fri, 28 May 2021 07:11:32 GMT
• Title: THINK: A Novel Conversation Model for Generating Grammatically Correct and Coherent Responses
• Authors: Bin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu and Kan Li
• Abstract summary: We propose a conversation model named "THINK" (Teamwork generation Hover around Impressive Noticeable Keywords) The model simplifies the context vectors and increases the coherence of generated responses in a reasonable way. Compared with other baselines, both automatic and human evaluation showed the advantages of our model.
• Score: 10.910845951559388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many existing conversation models that are based on the encoder-decoder framework have focused on ways to make the encoder more complicated to enrich the context vectors so as to increase the diversity and informativeness of generated responses. However, these approaches face two problems. First, the decoder is too simple to effectively utilize the previously generated information and tends to generate duplicated and self-contradicting responses. Second, the complex encoder tends to generate diverse but incoherent responses because the complex context vectors may deviate from the original semantics of context. In this work, we proposed a conversation model named "THINK" (Teamwork generation Hover around Impressive Noticeable Keywords) to make the decoder more complicated and avoid generating duplicated and self-contradicting responses. The model simplifies the context vectors and increases the coherence of generated responses in a reasonable way. For this model, we propose Teamwork generation framework and Semantics Extractor. Compared with other baselines, both automatic and human evaluation showed the advantages of our model.

Related papers

• Triple-Encoders: Representations That Fire Together, Wire Together [51.15206713482718]
  Contrastive Learning is a representation learning method that encodes relative distances between utterances into the embedding space via a bi-encoder. This study introduces triple-encoders, which efficiently compute distributed utterance mixtures from these independently encoded utterances. We find that triple-encoders lead to a substantial improvement over bi-encoders, and even to better zero-shot generalization than single-vector representation models.
  arXiv  Detail & Related papers  (2024-02-19T18:06:02Z)
• Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder [75.03283861464365]
  The seq2seq task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by an encoder to encode the source sequence and a decoder to generate the target text. Recently, a bunch of new approaches have emerged that apply decoder-only language models directly to the seq2seq task.
  arXiv  Detail & Related papers  (2023-04-08T15:44:29Z)
• X-ReCoSa: Multi-Scale Context Aggregation For Multi-Turn Dialogue Generation [0.0]
  In multi-turn dialogue generation, responses are not only related to the topic and background of the context but also related to words and phrases in the sentences of the context. Currently widely used hierarchical dialog models solely rely on context representations from the utterance-level encoder, ignoring the sentence representations output by the word-level encoder. We propose a new dialog model X-ReCoSa to tackle this problem which aggregates multi-scale context information for hierarchical dialog models.
  arXiv  Detail & Related papers  (2023-03-14T12:15:52Z)
• Twist Decoding: Diverse Generators Guide Each Other [116.20780037268801]
  We introduce Twist decoding, a simple and general inference algorithm that generates text while benefiting from diverse models. Our method does not assume the vocabulary, tokenization or even generation order is shared.
  arXiv  Detail & Related papers  (2022-05-19T01:27:53Z)
• A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation [13.820298189734686]
  This paper presents a novel open-domain dialogue generation model emphasizing the differentiation of speakers in multi-turn conversations. Our empirical results show that PHAED outperforms the state-of-the-art in both automatic and human evaluations.
  arXiv  Detail & Related papers  (2021-10-13T16:08:29Z)
• Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ? [41.36218215755317]
  We showcase evaluating the text generated through human or automatic metrics is not sufficient to appropriately evaluate soundness of the language understanding of dialogue models. We propose a set of probe tasks to evaluate encoder representation of different language encoders commonly used in dialogue models.
  arXiv  Detail & Related papers  (2021-06-20T04:52:37Z)
• Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems [53.160025961101354]
  We propose to train the generation model in a bidirectional manner by adding a backward reasoning step to the vanilla encoder-decoder training. The proposed backward reasoning step pushes the model to produce more informative and coherent content. Our method can improve response quality without introducing side information.
  arXiv  Detail & Related papers  (2021-04-30T20:38:27Z)
• Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension [49.92173751203827]
  In multi-turn dialog, utterances do not always take the full form of sentences. We propose to improve the response generation performance by examining the model's ability to answer a reading comprehension question.
  arXiv  Detail & Related papers  (2020-12-14T10:58:01Z)
• Predict and Use Latent Patterns for Short-Text Conversation [5.757975605648179]
  We propose to use more detailed semantic forms, including latent responses and part-of-speech sequences, as the controllable semantics to guide the generation. Our results show that the richer semantics are not only able to provide informative and diverse responses, but also increase the overall performance of response quality.
  arXiv  Detail & Related papers  (2020-10-27T01:31:42Z)
• Cascaded Text Generation with Markov Transformers [122.76100449018061]
  Two dominant approaches to neural text generation are fully autoregressive models, using serial beam search decoding, and non-autoregressive models, using parallel decoding with no output dependencies. This work proposes an autoregressive model with sub-linear parallel time generation. Noting that conditional random fields with bounded context can be decoded in parallel, we propose an efficient cascaded decoding approach for generating high-quality output. This approach requires only a small modification from standard autoregressive training, while showing competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
  arXiv  Detail & Related papers  (2020-06-01T17:52:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.