Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition

• URL: http://arxiv.org/abs/2107.10523v1
• Date: Thu, 22 Jul 2021 08:48:34 GMT
• Title: Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition
• Authors: Ying Zhang, Fandong Meng, Yufeng Chen, Jinan Xu, and Jie Zhou
• Abstract summary: We propose four practical guidelines to guide knowledge transfer and task fine-tuning. Based on these guidelines, we design a target-oriented fine-tuning (TOF) framework to exploit various data from three aspects in a unified training manner.
• Score: 25.662899487595524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-resource named entity recognition (NER) severely suffers from data scarcity in a specific domain or language. Most studies on zero-resource NER transfer knowledge from various data by fine-tuning on different auxiliary tasks. However, how to properly select training data and fine-tuning tasks is still an open problem. In this paper, we tackle the problem by transferring knowledge from three aspects, i.e., domain, language and task, and strengthening connections among them. Specifically, we propose four practical guidelines to guide knowledge transfer and task fine-tuning. Based on these guidelines, we design a target-oriented fine-tuning (TOF) framework to exploit various data from three aspects in a unified training manner. Experimental results on six benchmarks show that our method yields consistent improvements over baselines in both cross-domain and cross-lingual scenarios. Particularly, we achieve new state-of-the-art performance on five benchmarks.

Related papers

• IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition using Knowledge Bases [53.054598423181844]
  We present a novel NER cascade approach comprising three steps. We empirically demonstrate the significance of external knowledge bases in accurately classifying fine-grained and emerging entities. Our system exhibits robust performance in the MultiCoNER2 shared task, even in the low-resource language setting.
  arXiv  Detail & Related papers  (2023-04-20T20:30:34Z)
• Compositional Zero-Shot Domain Transfer with Text-to-Text Models [65.32821642379066]
  We propose a novel compositional transfer learning framework (DoT5) for zero-shot domain transfer. Without access to in-domain labels, DoT5 jointly learns domain knowledge and task knowledge in a multi-task manner. DoT5 demonstrates the effectiveness of compositional transfer learning through multi-task learning. In particular, DoT5 outperforms the current SOTA in zero-shot transfer by over 7 absolute points in accuracy on RadNLI.
  arXiv  Detail & Related papers  (2023-03-23T15:58:41Z)
• Characterization of effects of transfer learning across domains and languages [0.0]
  Transfer learning (TL) from pre-trained neural language models has emerged as a powerful technique over the years. We investigate how TL affects the performance of popular pre-trained models over three natural language processing (NLP) tasks.
  arXiv  Detail & Related papers  (2022-10-03T17:17:07Z)
• Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework [23.501276952950366]
  We propose a novel NER framework, namely SpanNER, which learns from natural language supervision and enables the identification of never-seen entity classes. We perform extensive experiments on 5 benchmark datasets and evaluate the proposed method in the few-shot learning, domain transfer and zero-shot learning settings. The experimental results show that the proposed method can bring 10%, 23% and 26% improvements in average over the best baselines in few-shot learning, domain transfer and zero-shot learning settings respectively.
  arXiv  Detail & Related papers  (2021-09-11T19:52:09Z)
• SSDL: Self-Supervised Domain Learning for Improved Face Recognition [0.0]
  Face recognition in unconstrained environments is challenging due to variations in illumination, quality of sensing, motion blur and etc. We propose a self-supervised domain learning scheme that trains on triplets mined from unlabelled data.
  arXiv  Detail & Related papers  (2020-11-26T15:55:59Z)
• Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering [80.60605604261416]
  We propose a novel neuro-symbolic framework for zero-shot question answering across commonsense tasks. We vary the set of language models, training regimes, knowledge sources, and data generation strategies, and measure their impact across tasks. We show that, while an individual knowledge graph is better suited for specific tasks, a global knowledge graph brings consistent gains across different tasks.
  arXiv  Detail & Related papers  (2020-11-07T22:52:21Z)
• Unsupervised Transfer Learning with Self-Supervised Remedy [60.315835711438936]
  Generalising deep networks to novel domains without manual labels is challenging to deep learning. Pre-learned knowledge does not transfer well without making strong assumptions about the learned and the novel domains. In this work, we aim to learn a discriminative latent space of the unlabelled target data in a novel domain by knowledge transfer from labelled related domains.
  arXiv  Detail & Related papers  (2020-06-08T16:42:17Z)
• An Iterative Multi-Knowledge Transfer Network for Aspect-Based Sentiment Analysis [73.7488524683061]
  We propose a novel Iterative Multi-Knowledge Transfer Network (IMKTN) for end-to-end ABSA. Our IMKTN transfers the task-specific knowledge from any two of the three subtasks to another one at the token level by utilizing a well-designed routing algorithm. Experimental results on three benchmark datasets demonstrate the effectiveness and superiority of our approach.
  arXiv  Detail & Related papers  (2020-04-04T13:49:54Z)
• Domain Adaption for Knowledge Tracing [65.86619804954283]
  We propose a novel adaptable framework, namely knowledge tracing (AKT) to address the DAKT problem. For the first aspect, we incorporate the educational characteristics (e.g., slip, guess, question texts) based on the deep knowledge tracing (DKT) to obtain a good performed knowledge tracing model. For the second aspect, we propose and adopt three domain adaptation processes. First, we pre-train an auto-encoder to select useful source instances for target model training.
  arXiv  Detail & Related papers  (2020-01-14T15:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.