A Regularization-based Transfer Learning Method for Information Extraction via Instructed Graph Decoder

• URL: http://arxiv.org/abs/2403.00891v1
• Date: Fri, 1 Mar 2024 13:04:12 GMT
• Title: A Regularization-based Transfer Learning Method for Information Extraction via Instructed Graph Decoder
• Authors: Kedi Chen and Jie Zhou and Qin Chen and Shunyu Liu and Liang He
• Abstract summary: We propose a regularization-based transfer learning method for IE (TIE) via an instructed graph decoder. Specifically, we first construct an instruction pool for datasets from all well-known IE tasks, and then present an instructed graph decoder. In this way, the common knowledge shared with existing datasets can be learned and transferred to a new dataset with new labels.
• Score: 29.242560023747252
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Information extraction (IE) aims to extract complex structured information from the text. Numerous datasets have been constructed for various IE tasks, leading to time-consuming and labor-intensive data annotations. Nevertheless, most prevailing methods focus on training task-specific models, while the common knowledge among different IE tasks is not explicitly modeled. Moreover, the same phrase may have inconsistent labels in different tasks, which poses a big challenge for knowledge transfer using a unified model. In this study, we propose a regularization-based transfer learning method for IE (TIE) via an instructed graph decoder. Specifically, we first construct an instruction pool for datasets from all well-known IE tasks, and then present an instructed graph decoder, which decodes various complex structures into a graph uniformly based on corresponding instructions. In this way, the common knowledge shared with existing datasets can be learned and transferred to a new dataset with new labels. Furthermore, to alleviate the label inconsistency problem among various IE tasks, we introduce a task-specific regularization strategy, which does not update the gradients of two tasks with 'opposite direction'. We conduct extensive experiments on 12 datasets spanning four IE tasks, and the results demonstrate the great advantages of our proposed method

Related papers

• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• InLINE: Inner-Layer Information Exchange for Multi-task Learning on Heterogeneous Graphs [13.204120407041195]
  Heterogeneous graph is an important structure for modeling complex data in real-world scenarios. We propose the Inner-Layer Information Exchange model that facilitate fine-grained information exchanges within each graph layer. Our model effectively alleviates the significant performance drop on specific tasks caused by negative transfer.
  arXiv  Detail & Related papers  (2024-10-29T14:46:49Z)
• Mirror: A Universal Framework for Various Information Extraction Tasks [28.43708291298155]
  We propose a universal framework for various IE tasks, namely Mirror. We recast existing IE tasks as a multi-span cyclic graph extraction problem and devise a non-autoregressive graph decoding algorithm. Our model has decent compatibility and outperforms or reaches competitive performance with SOTA systems.
  arXiv  Detail & Related papers  (2023-11-09T14:58:46Z)
• Universal Information Extraction with Meta-Pretrained Self-Retrieval [39.69130086395689]
  Universal Information Extraction(Universal IE) aims to solve different extraction tasks in a uniform text-to-structure generation manner. Retrieving knowledge from external knowledge bases may help models to overcome this problem but it is impossible to construct a knowledge base suitable for various IE tasks. We propose MetaRetriever to retrieve task-specific knowledge from PLMs to enhance universal IE.
  arXiv  Detail & Related papers  (2023-06-18T00:16:00Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• Easy-to-Hard Learning for Information Extraction [57.827955646831526]
  Information extraction systems aim to automatically extract structured information from unstructured texts. We propose a unified easy-to-hard learning framework consisting of three stages, i.e., the easy stage, the hard stage, and the main stage. By breaking down the learning process into multiple stages, our framework facilitates the model to acquire general IE task knowledge and improve its generalization ability.
  arXiv  Detail & Related papers  (2023-05-16T06:04:14Z)
• Gradient Imitation Reinforcement Learning for General Low-Resource Information Extraction [80.64518530825801]
  We develop a Gradient Reinforcement Learning (GIRL) method to encourage pseudo-labeled data to imitate the gradient descent direction on labeled data. We also leverage GIRL to solve all IE sub-tasks (named entity recognition, relation extraction, and event extraction) in low-resource settings.
  arXiv  Detail & Related papers  (2022-11-11T05:37:19Z)
• Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization [68.91386402390403]
  We propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during instruction learning. We conduct extensive experiments to show UDIT's effectiveness in various scenarios of tasks and datasets.
  arXiv  Detail & Related papers  (2022-10-17T15:25:24Z)
• Self-Supervised Graph Neural Network for Multi-Source Domain Adaptation [51.21190751266442]
  Domain adaptation (DA) tries to tackle the scenarios when the test data does not fully follow the same distribution of the training data. By learning from large-scale unlabeled samples, self-supervised learning has now become a new trend in deep learning. We propose a novel textbfSelf-textbfSupervised textbfGraph Neural Network (SSG) to enable more effective inter-task information exchange and knowledge sharing.
  arXiv  Detail & Related papers  (2022-04-08T03:37:56Z)
• Knowledge Graph Anchored Information-Extraction for Domain-Specific Insights [1.6308268213252761]
  We use a task-based approach for fulfilling specific information needs within a new domain. A pipeline constructed of state of the art NLP technologies is used to automatically extract an instance level semantic structure.
  arXiv  Detail & Related papers  (2021-04-18T19:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.