Rethinking Search: Making Experts out of Dilettantes

• URL: http://arxiv.org/abs/2105.02274v1
• Date: Wed, 5 May 2021 18:40:00 GMT
• Title: Rethinking Search: Making Experts out of Dilettantes
• Authors: Donald Metzler, Yi Tay, Dara Bahri, Marc Najork
• Abstract summary: When experiencing an information need, users want to engage with an expert, but often turn to an information retrieval system, such as a search engine. This paper examines how ideas from classical information retrieval and large pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of expert advice.
• Score: 55.90140165205178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When experiencing an information need, users want to engage with an expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Large pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than experts - they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and large pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of expert advice.

Related papers

• On the Biased Assessment of Expert Finding Systems [11.083396379885478]
  In large organisations, identifying experts on a given topic is crucial in leveraging the internal knowledge spread across teams and departments. This case study provides an analysis of how these recommendations can impact the evaluation of expert finding systems. We show that system-validated annotations lead to overestimated performance of traditional term-based retrieval models. We also augment knowledge areas with synonyms to uncover a strong bias towards literal mentions of their constituent words.
  arXiv  Detail & Related papers  (2024-10-07T13:19:08Z)
• Explaining Expert Search and Team Formation Systems with ExES [8.573682949137085]
  Expert search and team formation systems operate on collaboration networks. Given a keyword query corresponding to the desired skills, these systems identify experts that best match the query. We propose ExES, a tool designed to explain expert search and team formation systems using factual and counterfactual methods.
  arXiv  Detail & Related papers  (2024-05-21T15:53:35Z)
• Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation [22.38338205905379]
  We leverage reinforcement learning algorithms to overcome the above challenges by introducing a novel reward function. Our reward function combines an accuracy metric and a faithfulness metric to provide a balanced quality judgment of generated responses.
  arXiv  Detail & Related papers  (2023-11-02T02:42:41Z)
• Self-Knowledge Guided Retrieval Augmentation for Large Language Models [59.771098292611846]
  Large language models (LLMs) have shown superior performance without task-specific fine-tuning. Retrieval-based methods can offer non-parametric world knowledge and improve the performance on tasks such as question answering. Self-Knowledge guided Retrieval augmentation (SKR) is a simple yet effective method which can let LLMs refer to the questions they have previously encountered.
  arXiv  Detail & Related papers  (2023-10-08T04:22:33Z)
• Large Language Models for Information Retrieval: A Survey [58.30439850203101]
  Information retrieval has evolved from term-based methods to its integration with advanced neural models. Recent research has sought to leverage large language models (LLMs) to improve IR systems. We delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers.
  arXiv  Detail & Related papers  (2023-08-14T12:47:22Z)
• Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? [50.29862466940209]
  We introduce InfoSeek, a visual question answering dataset tailored for information-seeking questions. We analyze various pre-trained visual question answering models and gain insights into their characteristics. We show that accurate visual entity recognition can be used to improve performance on InfoSeek by retrieving relevant documents.
  arXiv  Detail & Related papers  (2023-02-23T00:33:54Z)
• Design Challenges for a Multi-Perspective Search Engine [44.48345943046946]
  We study a new perspective-oriented document retrieval paradigm. We discuss and assess the inherent natural language understanding challenges in order to achieve the goal. We use the prototype system to conduct a user survey in order to assess the utility of our paradigm.
  arXiv  Detail & Related papers  (2021-12-15T18:59:57Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• How Context Affects Language Models' Factual Predictions [134.29166998377187]
  We integrate information from a retrieval system with a pre-trained language model in a purely unsupervised way. We report that augmenting pre-trained language models in this way dramatically improves performance and that the resulting system, despite being unsupervised, is competitive with a supervised machine reading baseline.
  arXiv  Detail & Related papers  (2020-05-10T09:28:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.