CODER: An efficient framework for improving retrieval through COntextualized Document Embedding Reranking

• URL: http://arxiv.org/abs/2112.08766v1
• Date: Thu, 16 Dec 2021 10:25:26 GMT
• Title: CODER: An efficient framework for improving retrieval through COntextualized Document Embedding Reranking
• Authors: George Zerveas, Navid Rekabsaz, Daniel Cohen, Carsten Eickhoff
• Abstract summary: We present a framework for improving the performance of a wide class of retrieval models at minimal computational cost. It utilizes precomputed document representations extracted by a base dense retrieval method. It incurs a negligible computational overhead on top of any first-stage method at run time, allowing it to be easily combined with any state-of-the-art dense retrieval method.
• Score: 11.635294568328625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a framework for improving the performance of a wide class of retrieval models at minimal computational cost. It utilizes precomputed document representations extracted by a base dense retrieval method and involves training a model to jointly score a large set of retrieved candidate documents for each query, while potentially transforming on the fly the representation of each document in the context of the other candidates as well as the query itself. When scoring a document representation based on its similarity to a query, the model is thus aware of the representation of its "peer" documents. We show that our approach leads to substantial improvement in retrieval performance over the base method and over scoring candidate documents in isolation from one another, as in a pair-wise training setting. Crucially, unlike term-interaction rerankers based on BERT-like encoders, it incurs a negligible computational overhead on top of any first-stage method at run time, allowing it to be easily combined with any state-of-the-art dense retrieval method. Finally, concurrently considering a set of candidate documents for a given query enables additional valuable capabilities in retrieval, such as score calibration and mitigating societal biases in ranking.

Related papers

• Zero-Shot Listwise Document Reranking with a Large Language Model [58.64141622176841]
  We propose Listwise Reranker with a Large Language Model (LRL), which achieves strong reranking effectiveness without using any task-specific training data. Experiments on three TREC web search datasets demonstrate that LRL not only outperforms zero-shot pointwise methods when reranking first-stage retrieval results, but can also act as a final-stage reranker.
  arXiv  Detail & Related papers  (2023-05-03T14:45:34Z)
• CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion [68.19934563919192]
  We propose a curriculum sampling strategy that utilizes pseudo queries during training and progressively enhances the relevance between the generated query and the real query. Experimental results on both in-domain and out-of-domain datasets demonstrate that our approach outperforms previous dense retrieval models.
  arXiv  Detail & Related papers  (2022-12-18T15:57:46Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)
• Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation [49.940525611640346]
  Document Augmentation for dense Retrieval (DAR) framework augments the representations of documents with their Dense Augmentation and perturbations. We validate the performance of DAR on retrieval tasks with two benchmark datasets, showing that the proposed DAR significantly outperforms relevant baselines on the dense retrieval of both the labeled and unlabeled documents.
  arXiv  Detail & Related papers  (2022-03-15T09:07:38Z)
• Value Retrieval with Arbitrary Queries for Form-like Documents [50.5532781148902]
  We propose value retrieval with arbitrary queries for form-like documents. Our method predicts target value for an arbitrary query based on the understanding of layout and semantics of a form. We propose a simple document language modeling (simpleDLM) strategy to improve document understanding on large-scale model pre-training.
  arXiv  Detail & Related papers  (2021-12-15T01:12:02Z)
• A Proposed Conceptual Framework for a Representational Approach to Information Retrieval [42.67826268399347]
  This paper outlines a conceptual framework for understanding recent developments in information retrieval and natural language processing. I propose a representational approach that breaks the core text retrieval problem into a logical scoring model and a physical retrieval model.
  arXiv  Detail & Related papers  (2021-10-04T15:57:02Z)
• Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback [29.719150565643965]
  This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.
  arXiv  Detail & Related papers  (2021-08-30T18:10:26Z)
• Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval [11.465218502487959]
  We design a method to mimic the queries on each of the documents by an iterative clustering process. We also optimize the matching function with a two-step score calculation procedure. Experimental results on several popular ranking and QA datasets show that our model can achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2021-05-08T05:28:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.