YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction

• URL: http://arxiv.org/abs/2312.15548v3
• Date: Tue, 2 Apr 2024 09:36:35 GMT
• Title: YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction
• Authors: Xinglin Xiao, Yijie Wang, Nan Xu, Yuqi Wang, Hanxuan Yang, Minzheng Wang, Yin Luo, Lei Wang, Wenji Mao, Daniel Zeng,
• Abstract summary: We propose an end-to-end chat-enhanced instruction tuning framework for universal information extraction (YAYI-UIE) Specifically, we utilize dialogue data and information extraction data to enhance the information extraction performance jointly.
• Score: 20.32778991187863
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The difficulty of the information extraction task lies in dealing with the task-specific label schemas and heterogeneous data structures. Recent work has proposed methods based on large language models to uniformly model different information extraction tasks. However, these existing methods are deficient in their information extraction capabilities for Chinese languages other than English. In this paper, we propose an end-to-end chat-enhanced instruction tuning framework for universal information extraction (YAYI-UIE), which supports both Chinese and English. Specifically, we utilize dialogue data and information extraction data to enhance the information extraction performance jointly. Experimental results show that our proposed framework achieves state-of-the-art performance on Chinese datasets while also achieving comparable performance on English datasets under both supervised settings and zero-shot settings.

Related papers

• Improving Pinterest Search Relevance Using Large Language Models [15.24121687428178]
  We integrate Large Language Models (LLMs) into our search relevance model. Our approach uses search queries alongside content representations that include captions extracted from a generative visual language model. We distill from the LLM-based model into real-time servable model architectures and features.
  arXiv  Detail & Related papers  (2024-10-22T16:29:33Z)
• ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models [15.72709558213362]
  ChatUIE is an innovative unified information extraction framework built upon ChatGLM. reinforcement learning is employed to improve and align various tasks that involve confusing and limited samples. Our experimental results demonstrate that ChatUIE can significantly improve the performance of information extraction with a slight decrease in chatting ability.
  arXiv  Detail & Related papers  (2024-03-08T07:59:19Z)
• Instruct and Extract: Instruction Tuning for On-Demand Information Extraction [86.29491354355356]
  On-Demand Information Extraction aims to fulfill the personalized demands of real-world users. We present a benchmark named InstructIE, inclusive of both automatically generated training data, as well as the human-annotated test set. Building on InstructIE, we further develop an On-Demand Information Extractor, ODIE.
  arXiv  Detail & Related papers  (2023-10-24T17:54:25Z)
• From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models [6.520584613661788]
  We construct a Japanese instruction dataset by expanding and filtering existing datasets. We perform Low-Rank Adaptation (LoRA) tuning on both Japanese and English existing models.
  arXiv  Detail & Related papers  (2023-09-07T00:14:37Z)
• Unified Model Learning for Various Neural Machine Translation [63.320005222549646]
  Existing machine translation (NMT) studies mainly focus on developing dataset-specific models. We propose a versatile'' model, i.e., the Unified Model Learning for NMT (UMLNMT) that works with data from different tasks. OurNMT results in substantial improvements over dataset-specific models with significantly reduced model deployment costs.
  arXiv  Detail & Related papers  (2023-05-04T12:21:52Z)
• Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
  A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. We design a simple but effective ensemble-based framework that combines various transfer learning techniques. We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
  arXiv  Detail & Related papers  (2023-01-22T18:22:55Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)
• Syntactic-GCN Bert based Chinese Event Extraction [2.3104000011280403]
  We propose an integrated framework to perform Chinese event extraction. The proposed approach is a multiple channel input neural framework that integrates semantic features and syntactic features. Experimental results show that the proposed method outperforms the benchmark approaches significantly.
  arXiv  Detail & Related papers  (2021-12-18T14:07:54Z)
• Zero-Shot Information Extraction as a Unified Text-to-Triple Translation [56.01830747416606]
  We cast a suite of information extraction tasks into a text-to-triple translation framework. We formalize the task as a translation between task-specific input text and output triples. We study the zero-shot performance of this framework on open information extraction.
  arXiv  Detail & Related papers  (2021-09-23T06:54:19Z)
• Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering [8.558954185502012]
  We propose a method to improve the Cross-lingual Question Answering performance without requiring additional annotated data. We report a new state-of-the-art on four multilingual datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr)
  arXiv  Detail & Related papers  (2020-10-23T20:09:01Z)
• Mixed-Lingual Pre-training for Cross-lingual Summarization [54.4823498438831]
  Cross-lingual Summarization aims at producing a summary in the target language for an article in the source language. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks like translation and monolingual tasks like masked language models. Our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
  arXiv  Detail & Related papers  (2020-10-18T00:21:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.