Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective Retrieval

• URL: http://arxiv.org/abs/2405.12801v2
• Date: Fri, 25 Oct 2024 01:45:44 GMT
• Title: Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective Retrieval
• Authors: Jonghyun Song, Cheyon Jin, Wenlong Zhao, Andrew McCallum, Jay-Yoon Lee,
• Abstract summary: A common retrieve-and-rerank paradigm involves retrieving relevant candidates from a broad set using a fast bi-encoder (BE) We propose the Comparing Multiple Candidates framework, which compares a query and multiple embeddings of similar candidates through shallow self-attention layers. Experimental results on the ZeSHEL dataset demonstrate that CMC, when plugged in between bi-encoders and cross-encoders as a seamless intermediate reranker, can effectively improve recall@k.
• Score: 36.445443789036226
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: A common retrieve-and-rerank paradigm involves retrieving relevant candidates from a broad set using a fast bi-encoder (BE), followed by applying expensive but accurate cross-encoders (CE) to a limited candidate set. However, relying on this small subset is often susceptible to error propagation from the bi-encoders, which limits the overall performance. To address these issues, we propose the Comparing Multiple Candidates (CMC) framework. CMC compares a query and multiple embeddings of similar candidates (i.e., neighbors) through shallow self-attention layers, delivering rich representations contextualized to each other. Furthermore, CMC is scalable enough to handle multiple comparisons simultaneously. For example, comparing ~10K candidates with CMC takes a similar amount of time as comparing 16 candidates with CE. Experimental results on the ZeSHEL dataset demonstrate that CMC, when plugged in between bi-encoders and cross-encoders as a seamless intermediate reranker (BE-CMC-CE), can effectively improve recall@k (+4.8%-p, +3.5%-p for R@16, R@64) compared to using only bi-encoders (BE-CE), with negligible slowdown (<7%). Additionally, to verify CMC's effectiveness as the final-stage reranker in improving top-1 accuracy, we conduct experiments on downstream tasks such as entity, passage, and dialogue ranking. The results indicate that CMC is not only faster (11x) but also often more effective than CE, with improved prediction accuracy in Wikipedia entity linking (+0.7%-p) and DSTC7 dialogue ranking (+3.3%-p).

Related papers

• CR-CTC: Consistency regularization on CTC for improved speech recognition [18.996929774821822]
  Connectionist Temporal Classification (CTC) is a widely used method for automatic speech recognition (ASR) However, it often falls short in recognition performance compared to transducer or systems combining CTC and attention-based encoder-decoder (CTC/AED) We propose the Consistency-Regularized CTC (CR-CTC), which enforces consistency between two CTC distributions obtained from different augmented views of the input speech mel-spectrogram.
  arXiv  Detail & Related papers  (2024-10-07T14:56:07Z)
• Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask [74.64216073678617]
  AMD performs parallel NAR inference within contiguous blocks of output labels concealed using attention masks. A beam search algorithm is designed to leverage a dynamic fusion of CTC, AR Decoder, and AMD probabilities. Experiments on the LibriSpeech-100hr corpus suggest the tripartite Decoder incorporating the AMD module produces a maximum decoding speed-up ratio of 1.73x.
  arXiv  Detail & Related papers  (2024-06-14T13:42:38Z)
• CEBin: A Cost-Effective Framework for Large-Scale Binary Code Similarity Detection [23.8834126695488]
  Binary code similarity detection (BCSD) is a fundamental technique for various application. We propose a cost-effective BCSD framework, CEBin, which fuses embedding-based and comparison-based approaches.
  arXiv  Detail & Related papers  (2024-02-29T03:02:07Z)
• Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization [60.91600465922932]
  We present an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoder. Our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods.
  arXiv  Detail & Related papers  (2022-10-23T00:32:04Z)
• Revisiting Code Search in a Two-Stage Paradigm [67.02322603435628]
  TOSS is a two-stage fusion code search framework. It first uses IR-based and bi-encoder models to efficiently recall a small number of top-k code candidates. It then uses fine-grained cross-encoders for finer ranking.
  arXiv  Detail & Related papers  (2022-08-24T02:34:27Z)
• Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking [57.42241521034744]
  We propose the concept of certified error control of candidate set pruning for relevance ranking. Our method successfully prunes the first-stage retrieved candidate sets to improve the second-stage reranking speed.
  arXiv  Detail & Related papers  (2022-05-19T16:00:13Z)
• Intermediate Loss Regularization for CTC-based Speech Recognition [58.33721897180646]
  We present a simple and efficient auxiliary loss function for automatic speech recognition (ASR) based on the connectionist temporal classification ( CTC) objective. We evaluate the proposed method on various corpora, reaching word error rate (WER) 9.9% on the WSJ corpus and character error rate (CER) 5.2% on the AISHELL-1 corpus respectively.
  arXiv  Detail & Related papers  (2021-02-05T15:01:03Z)
• Solving for multi-class using orthogonal coding matrices [0.0]
  Error correcting code (ECC) is a common method of generalizing binary to multi-class classification. Here we test two types of orthogonal ECCs on seven different datasets. We compare them with three other multi-class methods: 1 vs. 1, one-versus-the-rest and random ECCs.
  arXiv  Detail & Related papers  (2018-01-27T08:45:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.