Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?

• URL: http://arxiv.org/abs/2311.09175v2
• Date: Tue, 30 Apr 2024 15:52:08 GMT
• Title: Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?
• Authors: Minghan Li, Honglei Zhuang, Kai Hui, Zhen Qin, Jimmy Lin, Rolf Jagerman, Xuanhui Wang, Michael Bendersky,
• Abstract summary: We show that it is possible to improve the generalization of a strong neural ranker, by prompt engineering and aggregating the ranking results of each expanded query via fusion. Experiments on BEIR and TREC Deep Learning show that the nDCG@10 scores of both MonoT5 and RankT5 following these steps are improved.
• Score: 72.42500059688396
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Query expansion has been widely used to improve the search results of first-stage retrievers, yet its influence on second-stage, cross-encoder rankers remains under-explored. A recent work of Weller et al. [44] shows that current expansion techniques benefit weaker models such as DPR and BM25 but harm stronger rankers such as MonoT5. In this paper, we re-examine this conclusion and raise the following question: Can query expansion improve generalization of strong cross-encoder rankers? To answer this question, we first apply popular query expansion methods to state-of-the-art cross-encoder rankers and verify the deteriorated zero-shot performance. We identify two vital steps for cross-encoders in the experiment: high-quality keyword generation and minimal-disruptive query modification. We show that it is possible to improve the generalization of a strong neural ranker, by prompt engineering and aggregating the ranking results of each expanded query via fusion. Specifically, we first call an instruction-following language model to generate keywords through a reasoning chain. Leveraging self-consistency and reciprocal rank weighting, we further combine the ranking results of each expanded query dynamically. Experiments on BEIR and TREC Deep Learning 2019/2020 show that the nDCG@10 scores of both MonoT5 and RankT5 following these steps are improved, which points out a direction for applying query expansion to strong cross-encoder rankers.

Related papers

• GenCRF: Generative Clustering and Reformulation Framework for Enhanced Intent-Driven Information Retrieval [20.807374287510623]
  We propose GenCRF: a Generative Clustering and Reformulation Framework to capture diverse intentions adaptively. We show that GenCRF achieves state-of-the-art performance, surpassing previous query reformulation SOTAs by up to 12% on nDCG@10.
  arXiv  Detail & Related papers  (2024-09-17T05:59:32Z)
• Ranking-based Adaptive Query Generation for DETRs in Crowded Pedestrian Detection [49.27380156754935]
  We find that the number of DETRs' queries must be adjusted manually, otherwise, the performance would degrade to varying degrees. We propose Rank-based Adaptive Query Generation (RAQG) to alleviate the problem. Our method is simple and effective, which can be plugged into any DETRs to make it query-adaptive in theory.
  arXiv  Detail & Related papers  (2023-10-24T11:00:56Z)
• Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering [28.05138829730091]
  EAR first applies a query expansion model to generate a diverse set of queries, and then uses a query reranker to select the ones that could lead to better retrieval results. By connecting better the query expansion model and retriever, EAR significantly enhances a traditional sparse retrieval method, BM25.
  arXiv  Detail & Related papers  (2023-05-26T16:41:03Z)
• ReFIT: Relevance Feedback from a Reranker during Inference [109.33278799999582]
  Retrieve-and-rerank is a prevalent framework in neural information retrieval. We propose to leverage the reranker to improve recall by making it provide relevance feedback to the retriever at inference time.
  arXiv  Detail & Related papers  (2023-05-19T15:30:33Z)
• KEPR: Knowledge Enhancement and Plausibility Ranking for Generative Commonsense Question Answering [11.537283115693432]
  We propose a Knowledge Enhancement and Plausibility Ranking approach grounded on the Generate-Then-Rank pipeline architecture. Specifically, we expand questions in terms of Wiktionary commonsense knowledge of keywords, and reformulate them with normalized patterns. We develop an ELECTRA-based answer ranking model, where logistic regression is conducted during training, with the aim of approxing different levels of plausibility.
  arXiv  Detail & Related papers  (2023-05-15T04:58:37Z)
• Noise-Robust Dense Retrieval via Contrastive Alignment Post Training [89.29256833403167]
  Contrastive Alignment POst Training (CAPOT) is a highly efficient finetuning method that improves model robustness without requiring index regeneration. CAPOT enables robust retrieval by freezing the document encoder while the query encoder learns to align noisy queries with their unaltered root. We evaluate CAPOT noisy variants of MSMARCO, Natural Questions, and Trivia QA passage retrieval, finding CAPOT has a similar impact as data augmentation with none of its overhead.
  arXiv  Detail & Related papers  (2023-04-06T22:16:53Z)
• AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation [44.93777271276723]
  We propose two approaches that enable annotation-free and scalable training by creating pseudo querydocument pairs. The query extraction method involves selecting salient spans from the original document to generate pseudo queries. The transferred query generation method utilizes generation models trained for other NLP tasks, such as summarization, to produce pseudo queries.
  arXiv  Detail & Related papers  (2022-12-17T10:43:25Z)
• Optimizing Test-Time Query Representations for Dense Retrieval [34.61821330771046]
  TOUR improves query representations guided by test-time retrieval results. We leverage a cross-encoder re-ranker to provide fine-grained pseudo labels over retrieval results. TOUR consistently improves direct re-ranking by up to 2.0% while running 1.3-2.4x faster.
  arXiv  Detail & Related papers  (2022-05-25T11:39:42Z)
• Adversarial Retriever-Ranker for dense text retrieval [51.87158529880056]
  We present Adversarial Retriever-Ranker (AR2), which consists of a dual-encoder retriever plus a cross-encoder ranker. AR2 consistently and significantly outperforms existing dense retriever methods. This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%), TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%)
  arXiv  Detail & Related papers  (2021-10-07T16:41:15Z)
• CoSQA: 20,000+ Web Queries for Code Search and Question Answering [63.92224685262063]
  CoSQA dataset includes 20,604 labels for pairs of natural language queries and codes. We introduce a contrastive learning method dubbed CoCLR to enhance query-code matching. We show that evaluated on CodeXGLUE with the same CodeBERT model, training on CoSQA improves the accuracy of code question answering by 5.1%.
  arXiv  Detail & Related papers  (2021-05-27T15:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.