Task-aware Retrieval with Instructions

• URL: http://arxiv.org/abs/2211.09260v1
• Date: Wed, 16 Nov 2022 23:13:22 GMT
• Title: Task-aware Retrieval with Instructions
• Authors: Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, Wen-tau Yih
• Abstract summary: We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries. We present TART, a multi-task retrieval system trained on the diverse retrieval tasks with instructions. TART shows strong capabilities to adapt to a new task via instructions and advances the state of the art on two zero-shot retrieval benchmarks.
• Score: 91.87694020194316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries, making the system task-aware. We aim to develop a general-purpose task-aware retrieval systems using multi-task instruction tuning that can follow human-written instructions to find the best documents for a given query. To this end, we introduce the first large-scale collection of approximately 40 retrieval datasets with instructions, and present TART, a multi-task retrieval system trained on the diverse retrieval tasks with instructions. TART shows strong capabilities to adapt to a new task via instructions and advances the state of the art on two zero-shot retrieval benchmarks, BEIR and LOTTE, outperforming models up to three times larger. We further introduce a new evaluation setup to better reflect real-world scenarios, pooling diverse documents and tasks. In this setup, TART significantly outperforms competitive baselines, further demonstrating the effectiveness of guiding retrieval with instructions.

Related papers

• Instruction Embedding: Latent Representations of Instructions Towards Task Identification [20.327984896070053]
  For instructional data, the most important aspect is the task it represents, rather than the specific semantics and knowledge information. In this work, we introduce a new concept, instruction embedding, and construct Instruction Embedding Benchmark (IEB) for its training and evaluation.
  arXiv  Detail & Related papers  (2024-09-29T12:12:24Z)
• Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification [62.894790379098005]
  We propose a novel instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Instruct-ReID is the first exploration of a general ReID setting, where existing 6 ReID tasks can be viewed as special cases by assigning different instructions. We propose a novel baseline model, IRM, with an adaptive triplet loss to handle various retrieval tasks within a unified framework.
  arXiv  Detail & Related papers  (2024-05-28T03:35:46Z)
• FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions [71.5977045423177]
  We study the use of instructions in Information Retrieval systems. We introduce our dataset FollowIR, which contains a rigorous instruction evaluation benchmark. We show that it is possible for IR models to learn to follow complex instructions.
  arXiv  Detail & Related papers  (2024-03-22T14:42:29Z)
• I3: Intent-Introspective Retrieval Conditioned on Instructions [83.91776238599824]
  I3 is a unified retrieval system that performs Intent-Introspective retrieval across various tasks conditioned on Instructions without task-specific training. I3 incorporates a pluggable introspector in a parameter-isolated manner to comprehend specific retrieval intents. It utilizes extensive LLM-generated data to train I3 phase-by-phase, embodying two key designs: progressive structure pruning and drawback-based data refinement.
  arXiv  Detail & Related papers  (2023-08-19T14:17:57Z)
• Automated Few-shot Classification with Instruction-Finetuned Language Models [76.69064714392165]
  We show that AuT-Few outperforms state-of-the-art few-shot learning methods. We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
  arXiv  Detail & Related papers  (2023-05-21T21:50:27Z)
• InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction [25.14501269639789]
  InstructUIE is a unified information extraction framework based on instruction tuning. In this paper, we propose InstructUIE, a unified information extraction framework based on instruction tuning.
  arXiv  Detail & Related papers  (2023-04-17T09:00:50Z)
• CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems [56.302581679816775]
  This paper proposes Comprehensive Instruction (CINS) that exploits PLMs with task-specific instructions. We design a schema (definition, constraint, prompt) of instructions and their customized realizations for three important downstream tasks in ToD. Experiments are conducted on these ToD tasks in realistic few-shot learning scenarios with small validation data.
  arXiv  Detail & Related papers  (2021-09-10T03:23:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.