Universal Information Extraction with Meta-Pretrained Self-Retrieval

• URL: http://arxiv.org/abs/2306.10444v1
• Date: Sun, 18 Jun 2023 00:16:00 GMT
• Title: Universal Information Extraction with Meta-Pretrained Self-Retrieval
• Authors: Xin Cong. Bowen Yu, Mengcheng Fang, Tingwen Liu, Haiyang Yu, Zhongkai Hu, Fei Huang, Yongbin Li, Bin Wang
• Abstract summary: Universal Information Extraction(Universal IE) aims to solve different extraction tasks in a uniform text-to-structure generation manner. Retrieving knowledge from external knowledge bases may help models to overcome this problem but it is impossible to construct a knowledge base suitable for various IE tasks. We propose MetaRetriever to retrieve task-specific knowledge from PLMs to enhance universal IE.
• Score: 39.69130086395689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal Information Extraction~(Universal IE) aims to solve different extraction tasks in a uniform text-to-structure generation manner. Such a generation procedure tends to struggle when there exist complex information structures to be extracted. Retrieving knowledge from external knowledge bases may help models to overcome this problem but it is impossible to construct a knowledge base suitable for various IE tasks. Inspired by the fact that large amount of knowledge are stored in the pretrained language models~(PLM) and can be retrieved explicitly, in this paper, we propose MetaRetriever to retrieve task-specific knowledge from PLMs to enhance universal IE. As different IE tasks need different knowledge, we further propose a Meta-Pretraining Algorithm which allows MetaRetriever to quicktly achieve maximum task-specific retrieval performance when fine-tuning on downstream IE tasks. Experimental results show that MetaRetriever achieves the new state-of-the-art on 4 IE tasks, 12 datasets under fully-supervised, low-resource and few-shot scenarios.

Related papers

• A Regularization-based Transfer Learning Method for Information Extraction via Instructed Graph Decoder [29.242560023747252]
  We propose a regularization-based transfer learning method for IE (TIE) via an instructed graph decoder. Specifically, we first construct an instruction pool for datasets from all well-known IE tasks, and then present an instructed graph decoder. In this way, the common knowledge shared with existing datasets can be learned and transferred to a new dataset with new labels.
  arXiv  Detail & Related papers  (2024-03-01T13:04:12Z)
• UMIE: Unified Multimodal Information Extraction with Instruction Tuning [12.777967562175437]
  We propose UMIE, a unified multimodal information extractor, to unify three MIE tasks as a generation problem using instruction tuning. Extensive experiments show that our single UMIE outperforms various state-of-the-art (SoTA) methods across six MIE datasets on three tasks. Our research serves as an initial step towards a unified MIE model and initiates the exploration into both instruction tuning and large language models within the MIE domain.
  arXiv  Detail & Related papers  (2024-01-05T22:52:15Z)
• Large Language Models for Generative Information Extraction: A Survey [89.71273968283616]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in text understanding and generation. We present an extensive overview by categorizing these works in terms of various IE subtasks and techniques. We empirically analyze the most advanced methods and discover the emerging trend of IE tasks with LLMs.
  arXiv  Detail & Related papers  (2023-12-29T14:25:22Z)
• Mirror: A Universal Framework for Various Information Extraction Tasks [28.43708291298155]
  We propose a universal framework for various IE tasks, namely Mirror. We recast existing IE tasks as a multi-span cyclic graph extraction problem and devise a non-autoregressive graph decoding algorithm. Our model has decent compatibility and outperforms or reaches competitive performance with SOTA systems.
  arXiv  Detail & Related papers  (2023-11-09T14:58:46Z)
• Easy-to-Hard Learning for Information Extraction [57.827955646831526]
  Information extraction systems aim to automatically extract structured information from unstructured texts. We propose a unified easy-to-hard learning framework consisting of three stages, i.e., the easy stage, the hard stage, and the main stage. By breaking down the learning process into multiple stages, our framework facilitates the model to acquire general IE task knowledge and improve its generalization ability.
  arXiv  Detail & Related papers  (2023-05-16T06:04:14Z)
• CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors [92.17328076003628]
  Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. In this paper, we propose to recast the structured output in the form of code instead of natural language.
  arXiv  Detail & Related papers  (2023-05-09T18:40:31Z)
• Unified Structure Generation for Universal Information Extraction [58.89057387608414]
  UIE can universally model different IE tasks, adaptively generate targeted structures, and collaboratively learn general IE abilities from different knowledge sources. Experiments show that UIE achieved the state-of-the-art performance on 4 IE tasks, 13 datasets, and on all supervised, low-resource, and few-shot settings.
  arXiv  Detail & Related papers  (2022-03-23T08:49:29Z)
• Automated Relational Meta-learning [95.02216511235191]
  We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph. We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-01-03T07:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.