Fine-grained Conversational Decoding via Isotropic and Proximal Search

• URL: http://arxiv.org/abs/2310.08130v4
• Date: Wed, 15 Nov 2023 03:59:42 GMT
• Title: Fine-grained Conversational Decoding via Isotropic and Proximal Search
• Authors: Yuxuan Yao, Han Wu, Qiling Xu, Linqi Song
• Abstract summary: We present a fine-grained conversational decoding method, termed textitisotropic and proximal search (IPS). Our method is designed to generate the semantic-concentrated response, while still maintaining informativeness and discrimination against the context. Experiments show that our approach outperforms existing decoding strategies in the dialogue field across both automatic and human evaluation metrics.
• Score: 17.904421465456814
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: General-purpose text decoding approaches are usually adopted for dialogue response generation. Although the quality of the generated responses can be improved with dialogue-specific encoding methods, conversational decoding methods are still under-explored. Inspired by \citet{wu2023learning} that a good dialogue feature space should follow the rules of locality and isotropy, we present a fine-grained conversational decoding method, termed \textit{isotropic and proximal search (IPS)}. Our method is designed to generate the semantic-concentrated response, while still maintaining informativeness and discrimination against the context. Experiments show that our approach outperforms existing decoding strategies in the dialogue field across both automatic and human evaluation metrics. More in-depth analyses further confirm the effectiveness of our approach.

Related papers

• Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog [83.63849872250651]
  Video-grounded dialog requires profound understanding of both dialog history and video content for accurate response generation. We present an iterative search and reasoning framework, which consists of a textual encoder, a visual encoder, and a generator.
  arXiv  Detail & Related papers  (2023-10-11T07:37:13Z)
• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems [22.302137281411646]
  Dial-MAE is a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks.
  arXiv  Detail & Related papers  (2023-06-07T11:40:07Z)
• Revisiting Conversation Discourse for Dialogue Disentanglement [88.3386821205896]
  We propose enhancing dialogue disentanglement by taking full advantage of the dialogue discourse characteristics. We develop a structure-aware framework to integrate the rich structural features for better modeling the conversational semantic context. Our work has great potential to facilitate broader multi-party multi-thread dialogue applications.
  arXiv  Detail & Related papers  (2023-06-06T19:17:47Z)
• Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information [18.859159491548006]
  We propose a novel learning-based automatic evaluation metric (CMN) for open-domain dialogues. We employ Conditional Variational Autoencoders (CVAEs) with a Next Sentence Prediction (NSP) objective and employing Mutual Information (MI) to model the semantic similarity of text in the latent space. Experimental results on two open-domain dialogue datasets demonstrate the superiority of our method compared with a wide range of baselines.
  arXiv  Detail & Related papers  (2023-05-26T14:21:54Z)
• Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs [59.74002011562726]
  We propose a novel linguistic cue-based chain-of-thoughts (textitCue-CoT) to provide a more personalized and engaging response. We build a benchmark with in-depth dialogue questions, consisting of 6 datasets in both Chinese and English. Empirical results demonstrate our proposed textitCue-CoT method outperforms standard prompting methods in terms of both textithelpfulness and textitacceptability on all datasets.
  arXiv  Detail & Related papers  (2023-05-19T16:27:43Z)
• Improve Retrieval-based Dialogue System via Syntax-Informed Attention [46.79601705850277]
  We propose SIA, Syntax-Informed Attention, considering both intra- and inter-sentence syntax information. We evaluate our method on three widely used benchmarks and experimental results demonstrate the general superiority of our method on dialogue response selection.
  arXiv  Detail & Related papers  (2023-03-12T08:14:16Z)
• An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation [27.445562543667357]
  We propose an Equal-size Hard Expectation--Maximization algorithm to train a multi-decoder model for diverse dialogue generation. Our algorithm assigns a sample to a decoder in a hard manner and additionally imposes an equal-assignment constraint to ensure that all decoders are well-trained.
  arXiv  Detail & Related papers  (2022-09-29T08:41:32Z)
• Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection [37.15893335147598]
  A limitation of current neural dialog models is that they tend to suffer from a lack of specificity and informativeness in generated responses. We propose a post-hoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model. We construct multiple candidate responses, individually injecting each retrieved snippet into the initial response using a gradient-based decoding method, and then select the final response with an unsupervised ranking step.
  arXiv  Detail & Related papers  (2022-03-22T00:42:27Z)
• Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems [53.160025961101354]
  We propose to train the generation model in a bidirectional manner by adding a backward reasoning step to the vanilla encoder-decoder training. The proposed backward reasoning step pushes the model to produce more informative and coherent content. Our method can improve response quality without introducing side information.
  arXiv  Detail & Related papers  (2021-04-30T20:38:27Z)
• Stylized Dialogue Response Generation Using Stylized Unpaired Texts [63.69880979112312]
  This paper proposes a stylized dialogue generation method that can capture stylistic features embedded in unpaired texts. Our method can produce dialogue responses that are both coherent to the given context and conform to the target style.
  arXiv  Detail & Related papers  (2020-09-27T01:04:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.