Towards Robust Ranker for Text Retrieval

• URL: http://arxiv.org/abs/2206.08063v1
• Date: Thu, 16 Jun 2022 10:27:46 GMT
• Title: Towards Robust Ranker for Text Retrieval
• Authors: Yucheng Zhou, Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Guodong Long, Binxing Jiao, Daxin Jiang
• Abstract summary: A ranker plays an indispensable role in the de facto'retrieval & rerank' pipeline. A ranker plays an indispensable role in the de facto'retrieval & rerank' pipeline.
• Score: 83.15191578888188
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A ranker plays an indispensable role in the de facto 'retrieval & rerank' pipeline, but its training still lags behind -- learning from moderate negatives or/and serving as an auxiliary module for a retriever. In this work, we first identify two major barriers to a robust ranker, i.e., inherent label noises caused by a well-trained retriever and non-ideal negatives sampled for a high-capable ranker. Thereby, we propose multiple retrievers as negative generators improve the ranker's robustness, where i) involving extensive out-of-distribution label noises renders the ranker against each noise distribution, and ii) diverse hard negatives from a joint distribution are relatively close to the ranker's negative distribution, leading to more challenging thus effective training. To evaluate our robust ranker (dubbed R$^2$anker), we conduct experiments in various settings on the popular passage retrieval benchmark, including BM25-reranking, full-ranking, retriever distillation, etc. The empirical results verify the new state-of-the-art effectiveness of our model.

Related papers

• Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers [6.773411876899064]
  inference-free sparse models lag far behind in terms of search relevance when compared to both sparse and dense siamese models. We propose two different approaches for performance improvement. First, we introduce the IDF-aware FLOPS loss, which introduces Inverted Document Frequency (IDF) to the sparsification of representations. We find that it mitigates the negative impact of the FLOPS regularization on search relevance, allowing the model to achieve a better balance between accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-11-07T03:46:43Z)
• ReFIT: Relevance Feedback from a Reranker during Inference [109.33278799999582]
  Retrieve-and-rerank is a prevalent framework in neural information retrieval. We propose to leverage the reranker to improve recall by making it provide relevance feedback to the retriever at inference time.
  arXiv  Detail & Related papers  (2023-05-19T15:30:33Z)
• HYRR: Hybrid Infused Reranking for Passage Retrieval [18.537666294601458]
  Hybrid Infused Reranking for Passages Retrieval is a framework for training rerankers based on a hybrid of BM25 and neural retrieval models. We present evaluations on a supervised passage retrieval task using MS MARCO and zero-shot retrieval tasks using BEIR.
  arXiv  Detail & Related papers  (2022-12-20T18:44:21Z)
• LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval [68.85686621130111]
  We propose to make a dense retriever align a well-performing lexicon-aware representation model. We evaluate our model on three public benchmarks, which shows that with a comparable lexicon-aware retriever as the teacher, our proposed dense model can bring consistent and significant improvements.
  arXiv  Detail & Related papers  (2022-08-29T15:09:28Z)
• Cooperative Retriever and Ranker in Deep Recommenders [75.35463122701135]
  Deep recommender systems (DRS) are intensively applied in modern web services. DRS employs a two-stage workflow: retrieval and ranking, to generate its recommendation results. It remains to explore effective collaborations between retriever and ranker.
  arXiv  Detail & Related papers  (2022-06-28T03:41:50Z)
• Adversarial Retriever-Ranker for dense text retrieval [51.87158529880056]
  We present Adversarial Retriever-Ranker (AR2), which consists of a dual-encoder retriever plus a cross-encoder ranker. AR2 consistently and significantly outperforms existing dense retriever methods. This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%), TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%)
  arXiv  Detail & Related papers  (2021-10-07T16:41:15Z)
• Combating False Negatives in Adversarial Imitation Learning [67.99941805086154]
  In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior. As the trained policy learns to be more successful, the negative examples become increasingly similar to expert ones. We propose a method to alleviate the impact of false negatives and test it on the BabyAI environment.
  arXiv  Detail & Related papers  (2020-02-02T14:56:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.