InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction

• URL: http://arxiv.org/abs/2304.08085v1
• Date: Mon, 17 Apr 2023 09:00:50 GMT
• Title: InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction
• Authors: Xiao Wang, Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, Jihua Kang, Jingsheng Yang, Siyuan Li, Chunsai Du
• Abstract summary: InstructUIE is a unified information extraction framework based on instruction tuning. In this paper, we propose InstructUIE, a unified information extraction framework based on instruction tuning.
• Score: 25.14501269639789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models have unlocked strong multi-task capabilities from reading instructive prompts. However, recent studies have shown that existing large models still have difficulty with information extraction tasks. For example, gpt-3.5-turbo achieved an F1 score of 18.22 on the Ontonotes dataset, which is significantly lower than the state-of-the-art performance. In this paper, we propose InstructUIE, a unified information extraction framework based on instruction tuning, which can uniformly model various information extraction tasks and capture the inter-task dependency. To validate the proposed method, we introduce IE INSTRUCTIONS, a benchmark of 32 diverse information extraction datasets in a unified text-to-text format with expert-written instructions. Experimental results demonstrate that our method achieves comparable performance to Bert in supervised settings and significantly outperforms the state-of-the-art and gpt3.5 in zero-shot settings.

Related papers

• INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning [59.07490387145391]
  Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks. Their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. We introduce a novel instruction tuning dataset, INTERS, encompassing 20 tasks across three fundamental IR categories.
  arXiv  Detail & Related papers  (2024-01-12T12:10:28Z)
• Automated Few-shot Classification with Instruction-Finetuned Language Models [76.69064714392165]
  We show that AuT-Few outperforms state-of-the-art few-shot learning methods. We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
  arXiv  Detail & Related papers  (2023-05-21T21:50:27Z)
• Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning [37.01833561948585]
  Recent works on instruction tuning (IT) have achieved great performance with zero-shot generalizability to unseen tasks. We analyze how models utilize instructions during IT by comparing model training with altered vs. original instructions.
  arXiv  Detail & Related papers  (2023-05-19T02:00:47Z)
• Unified Demonstration Retriever for In-Context Learning [56.06473069923567]
  Unified Demonstration Retriever (textbfUDR) is a single model to retrieve demonstrations for a wide range of tasks. We propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates. Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines.
  arXiv  Detail & Related papers  (2023-05-07T16:07:11Z)
• MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning [24.741736629886564]
  Instruction tuning is a new learning paradigm that fine-tunes pre-trained language models on tasks specified through instructions. We introduce MUL-TIINSTRUCT, the first multimodal instruction tuning benchmark dataset. We show strong zero-shot performance on various unseen multimodal tasks and the benefit of transfer learning from a text-only instruction dataset.
  arXiv  Detail & Related papers  (2022-12-21T05:17:06Z)
• Instruction Induction: From Few Examples to Natural Language Task Descriptions [55.139554327372934]
  We show that language models can explicitly infer an underlying task from a few demonstrations by prompting them to generate a natural language instruction that fits the examples. InstructGPT achieves 65.7% of human performance in our execution-based metric, while the original GPT-3 model reaches only 9.8% of human performance.
  arXiv  Detail & Related papers  (2022-05-22T09:22:37Z)
• How Many Data Samples is an Additional Instruction Worth? [20.66688303609522]
  Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Our results indicate that an additional instruction can be equivalent to 200 data samples on average across tasks.
  arXiv  Detail & Related papers  (2022-03-17T08:30:30Z)
• Reframing Instructional Prompts to GPTk's Language [72.69833640335519]
  We propose reframing techniques for model designers to create effective prompts for language models. Our results show that reframing improves few-shot learning performance by 14% while reducing sample complexity. The performance gains are particularly important on large language models, such as GPT3 where tuning models or prompts on large datasets is not feasible.
  arXiv  Detail & Related papers  (2021-09-16T09:44:43Z)
• Finetuned Language Models Are Zero-Shot Learners [67.70352207685558]
  We show that instruction tuning boosts zero-shot performance on unseen tasks. We take a 137B parameter pretrained language model and instruction-tune it on over 60 NLP tasks verbalized via natural language instruction templates. We evaluate this instruction-tuned model, which we call FLAN, on unseen task types.
  arXiv  Detail & Related papers  (2021-09-03T17:55:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.