Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations

• URL: http://arxiv.org/abs/2109.13087v1
• Date: Fri, 24 Sep 2021 08:22:35 GMT
• Title: Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
• Authors: Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang and Nan Duan
• Abstract summary: We propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. To evaluate the performance of our proposed model, we construct two new datasets based on the Reddit comments dump and Twitter corpus.
• Score: 48.046725390986595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of coarse-grained response selection in retrieval-based dialogue systems. The problem is equally important with fine-grained response selection, but is less explored in existing literature. In this paper, we propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. In our CFC model, dense representations of query, candidate response and corresponding context is learned based on the multi-tower architecture, and more expressive knowledge learned from the one-tower architecture (fine-grained) is distilled into the multi-tower architecture (coarse-grained) to enhance the performance of the retriever. To evaluate the performance of our proposed model, we construct two new datasets based on the Reddit comments dump and Twitter corpus. Extensive experimental results on the two datasets show that the proposed methods achieve a significant improvement over all evaluation metrics compared with traditional baseline methods.

Related papers

• Optimization of Retrieval-Augmented Generation Context with Outlier Detection [0.0]
  We focus on methods to reduce the size and improve the quality of the prompt context required for question-answering systems. Our goal is to select the most semantically relevant documents, treating the discarded ones as outliers. It was found that the greatest improvements were achieved with increasing complexity of the questions and answers.
  arXiv  Detail & Related papers  (2024-07-01T15:53:29Z)
• Selecting Query-bag as Pseudo Relevance Feedback for Information-seeking Conversations [76.70349332096693]
  Information-seeking dialogue systems are widely used in e-commerce systems. We propose a Query-bag based Pseudo Relevance Feedback framework (QB-PRF) It constructs a query-bag with related queries to serve as pseudo signals to guide information-seeking conversations.
  arXiv  Detail & Related papers  (2024-03-22T08:10:32Z)
• Revisiting Sparse Retrieval for Few-shot Entity Linking [33.15662306409253]
  We propose an ELECTRA-based keyword extractor to denoise the mention context and construct a better query expression. For training the extractor, we propose a distant supervision method to automatically generate training data based on overlapping tokens between mention contexts and entity descriptions. Experimental results on the ZESHEL dataset demonstrate that the proposed method outperforms state-of-the-art models by a significant margin across all test domains.
  arXiv  Detail & Related papers  (2023-10-19T03:51:10Z)
• Reranking Passages with Coarse-to-Fine Neural Retriever Enhanced by List-Context Information [0.9463895540925061]
  This paper presents a list-context attention mechanism to augment the passage representation by incorporating the list-context information from other candidates. The proposed coarse-to-fine (C2F) neural retriever addresses the out-of-memory limitation of the passage attention mechanism. It integrates the coarse and fine rankers into the joint optimization process, allowing for feedback between the two layers to update the model simultaneously.
  arXiv  Detail & Related papers  (2023-08-23T09:29:29Z)
• Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification [8.59563091603226]
  We propose Differentiable Retrieval Augmentation via Generative lANguage modeling(Dragan) to address this problem by a novel differentiable reformulation. We demonstrate the effectiveness of our proposed method on a challenging NLP task in e-commerce search, namely query intent classification.
  arXiv  Detail & Related papers  (2023-08-18T05:05:35Z)
• Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference [82.28542500317445]
  We present an end-to-end learning framework, termed Sequential Posterior Inference (SPI), capable of selecting knowledge and generating dialogues. Unlike other methods, SPI does not require the inference network or assume a simple geometry of the posterior distribution.
  arXiv  Detail & Related papers  (2023-06-01T21:23:13Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• AutoRC: Improving BERT Based Relation Classification Models via Architecture Search [50.349407334562045]
  BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models. No consensus can be reached on what is the optimal architecture. We design a comprehensive search space for BERT based RC models and employ neural architecture search (NAS) method to automatically discover the design choices.
  arXiv  Detail & Related papers  (2020-09-22T16:55:49Z)
• Tradeoffs in Sentence Selection Techniques for Open-Domain Question Answering [54.541952928070344]
  We describe two groups of models for sentence selection: QA-based approaches, which run a full-fledged QA system to identify answer candidates, and retrieval-based models, which find parts of each passage specifically related to each question. We show that very lightweight QA models can do well at this task, but retrieval-based models are faster still.
  arXiv  Detail & Related papers  (2020-09-18T23:39:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.