Dense Sparse Retrieval: Using Sparse Language Models for Inference
Efficient Dense Retrieval
- URL: http://arxiv.org/abs/2304.00114v1
- Date: Fri, 31 Mar 2023 20:21:32 GMT
- Title: Dense Sparse Retrieval: Using Sparse Language Models for Inference
Efficient Dense Retrieval
- Authors: Daniel Campos, ChengXiang Zhai
- Abstract summary: We study how sparse language models can be used for dense retrieval to improve inference efficiency.
We find that sparse language models can be used as direct replacements with little to no drop in accuracy and up to 4.3x improved inference speeds.
- Score: 37.22592489907125
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Vector-based retrieval systems have become a common staple for academic and
industrial search applications because they provide a simple and scalable way
of extending the search to leverage contextual representations for documents
and queries. As these vector-based systems rely on contextual language models,
their usage commonly requires GPUs, which can be expensive and difficult to
manage. Given recent advances in introducing sparsity into language models for
improved inference efficiency, in this paper, we study how sparse language
models can be used for dense retrieval to improve inference efficiency. Using
the popular retrieval library Tevatron and the MSMARCO, NQ, and TriviaQA
datasets, we find that sparse language models can be used as direct
replacements with little to no drop in accuracy and up to 4.3x improved
inference speeds
Related papers
- Enhancing Cloud-Based Large Language Model Processing with Elasticsearch
and Transformer Models [17.09116903102371]
Large Language Models (LLMs) are a class of generative AI models built using the Transformer network.
LLMs are capable of leveraging vast datasets to identify, summarize, translate, predict, and generate language.
Semantic vector search within large language models is a potent technique that can significantly enhance search result accuracy and relevance.
arXiv Detail & Related papers (2024-02-24T12:31:22Z) - Evaluating Embedding APIs for Information Retrieval [51.24236853841468]
We evaluate the capabilities of existing semantic embedding APIs on domain generalization and multilingual retrieval.
We find that re-ranking BM25 results using the APIs is a budget-friendly approach and is most effective in English.
For non-English retrieval, re-ranking still improves the results, but a hybrid model with BM25 works best, albeit at a higher cost.
arXiv Detail & Related papers (2023-05-10T16:40:52Z) - BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer [1.911678487931003]
Retrieval-based language models are increasingly employed in question-answering tasks.
We develop the first Norwegian retrieval-based model by adapting the REALM framework.
We show that this type of training improves the reader's performance on extractive question-answering.
arXiv Detail & Related papers (2023-04-19T13:40:47Z) - UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
Large-scale retrieval is to recall relevant documents from a huge collection given a query.
Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms.
We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
arXiv Detail & Related papers (2022-05-23T11:01:59Z) - Autoregressive Search Engines: Generating Substrings as Document
Identifiers [53.0729058170278]
Autoregressive language models are emerging as the de-facto standard for generating answers.
Previous work has explored ways to partition the search space into hierarchical structures.
In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
arXiv Detail & Related papers (2022-04-22T10:45:01Z) - Efficient Nearest Neighbor Language Models [114.40866461741795]
Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore.
We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
arXiv Detail & Related papers (2021-09-09T12:32:28Z) - Comparison of Interactive Knowledge Base Spelling Correction Models for
Low-Resource Languages [81.90356787324481]
Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict.
This work shows a comparison of a neural model and character language models with varying amounts on target language data.
Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
arXiv Detail & Related papers (2020-10-20T17:31:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.