ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER

• URL: http://arxiv.org/abs/2305.17951v1
• Date: Mon, 29 May 2023 08:24:42 GMT
• Title: ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER
• Authors: Amirhossein Layegh, Amir H. Payberah, Ahmet Soylu, Dumitru Roman, Mihhail Matskin
• Abstract summary: We present ContrastNER, a prompt-based NER framework that employs both discrete and continuous tokens in prompts and uses a contrastive learning approach to learn the continuous prompts and forecast entity types. The experimental results demonstrate that ContrastNER obtains competitive performance to the state-of-the-art NER methods in high-resource settings and outperforms the state-of-the-art models in low-resource circumstances without requiring extensive prompt engineering and verbalizer design.
• Score: 0.6562256987706128
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Prompt-based language models have produced encouraging results in numerous applications, including Named Entity Recognition (NER) tasks. NER aims to identify entities in a sentence and provide their types. However, the strong performance of most available NER approaches is heavily dependent on the design of discrete prompts and a verbalizer to map the model-predicted outputs to entity categories, which are complicated undertakings. To address these challenges, we present ContrastNER, a prompt-based NER framework that employs both discrete and continuous tokens in prompts and uses a contrastive learning approach to learn the continuous prompts and forecast entity types. The experimental results demonstrate that ContrastNER obtains competitive performance to the state-of-the-art NER methods in high-resource settings and outperforms the state-of-the-art models in low-resource circumstances without requiring extensive manual prompt engineering and verbalizer design.

Related papers

• Show Less, Instruct More: Enriching Prompts with Definitions and Guidelines for Zero-Shot NER [3.4998124138877786]
  We propose SLIMER, an approach designed to tackle never-seen-before entity tags by instructing the model on fewer examples. Experiments demonstrate that definition and guidelines yield better performance, faster and more robust learning.
  arXiv  Detail & Related papers  (2024-07-01T13:25:33Z)
• Mix of Experts Language Model for Named Entity Recognition [4.120505838411977]
  We propose a robust NER model named BOND-MoE based on Mixture of Experts (MoE) Instead of relying on a single model for NER prediction, multiple models are trained and ensembled under the Expectation-Maximization framework. Experiments on real-world datasets show that the proposed method achieves state-of-the-art performance compared with other distantly supervised NER.
  arXiv  Detail & Related papers  (2024-04-30T01:41:03Z)
• ToNER: Type-oriented Named Entity Recognition with Generative Language Model [14.11486479935094]
  We propose a novel NER framework, namely ToNER based on a generative model. In ToNER, a type matching model is proposed at first to identify the entity types most likely to appear in the sentence. We append a multiple binary classification task to fine-tune the generative model's encoder, so as to generate the refined representation of the input sentence.
  arXiv  Detail & Related papers  (2024-04-14T05:13:37Z)
• SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entities [10.193908215351497]
  We propose SCANNER, a model capable of effectively handling all three NER variants. SCANNER is a two-stage structure; we extract entity candidates in the first stage and use it as a query to get knowledge. To tackle the challenges arising from noisy annotations in NER datasets, we introduce a novel self-distillation method.
  arXiv  Detail & Related papers  (2024-04-02T13:05:41Z)
• In-Context Learning for Few-Shot Nested Named Entity Recognition [53.55310639969833]
  We introduce an effective and innovative ICL framework for the setting of few-shot nested NER. We improve the ICL prompt by devising a novel example demonstration selection mechanism, EnDe retriever. In EnDe retriever, we employ contrastive learning to perform three types of representation learning, in terms of semantic similarity, boundary similarity, and label similarity.
  arXiv  Detail & Related papers  (2024-02-02T06:57:53Z)
• Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach [50.12455129619845]
  Named Entity Recognition (NER) aims to extract and classify entity mentions in the text into pre-defined types. We propose to incorporate the label dependencies among entity types into a multi-task learning framework for better MRC-based NER.
  arXiv  Detail & Related papers  (2023-09-20T03:15:05Z)
• E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition [69.87816981427858]
  Most named entity recognition (NER) systems focus on improving model performance, ignoring the need to quantify model uncertainty. Evidential deep learning (EDL) has recently been proposed as a promising solution to explicitly model predictive uncertainty for classification tasks. We propose a trustworthy NER framework named E-NER by introducing two uncertainty-guided loss terms to the conventional EDL, along with a series of uncertainty-guided training strategies.
  arXiv  Detail & Related papers  (2023-05-29T02:36:16Z)
• PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search [56.81939214465558]
  We propose PromptNER: a novel prompting method for few-shot NER via k nearest neighbor search. We use prompts that contains entity category information to construct label prototypes, which enables our model to fine-tune with only the support set. Our approach achieves excellent transfer learning ability, and extensive experiments on the Few-NERD and CrossNER datasets demonstrate that our model achieves superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-20T15:47:59Z)
• MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective [57.19660234992812]
  NER model has achieved promising performance on standard NER benchmarks. Recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. We propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective.
  arXiv  Detail & Related papers  (2022-04-09T05:18:20Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.