CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning

• URL: http://arxiv.org/abs/2109.07589v1
• Date: Wed, 15 Sep 2021 21:41:16 GMT
• Title: CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning
• Authors: Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca J. Passonneau, Rui Zhang
• Abstract summary: Existing approaches only learn class-specific semantic features and intermediate representations from source domains. We present CONTaiNER, a novel contrastive learning technique that optimize the inter-token distribution distance for Few-Shot NER. Our experiments in several traditional test domains (OntoNotes, CoNLL'03, WNUT '17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that CONTaiNER outperforms previous methods by 3%-13% absolute F1 points.
• Score: 11.289324473201614
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains. Existing approaches only learn class-specific semantic features and intermediate representations from source domains. This affects generalizability to unseen target domains, resulting in suboptimal performances. To this end, we present CONTaiNER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few-Shot NER. Instead of optimizing class-specific attributes, CONTaiNER optimizes a generalized objective of differentiating between token categories based on their Gaussian-distributed embeddings. This effectively alleviates overfitting issues originating from training domains. Our experiments in several traditional test domains (OntoNotes, CoNLL'03, WNUT '17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that on average, CONTaiNER outperforms previous methods by 3%-13% absolute F1 points while showing consistent performance trends, even in challenging scenarios where previous approaches could not achieve appreciable performance.

Related papers

• FsPONER: Few-shot Prompt Optimization for Named Entity Recognition in Domain-specific Scenarios [0.5106912532044251]
  We introduce FsPONER, a novel approach for optimizing few-shot prompts, and evaluate its performance on domain-specific NER datasets. FsPONER consists of three few-shot selection methods based on random sampling, TF-IDF, and a combination of both. In the considered real-world scenarios with data scarcity, FsPONER with TF-IDF surpasses fine-tuned models by approximately 10% in F1 score.
  arXiv  Detail & Related papers  (2024-07-10T20:32:50Z)
• PromptNER: Prompting For Named Entity Recognition [27.501500279749475]
  We introduce PromptNER, a new state-of-the-art algorithm for few-Shot and cross-domain NER. PromptNER achieves a 4% (absolute) improvement in F1 score on the ConLL dataset, a 9% (absolute) improvement on the GENIA dataset, and a 4% (absolute) improvement on the FewNERD dataset.
  arXiv  Detail & Related papers  (2023-05-24T07:38:24Z)
• Self-training through Classifier Disagreement for Cross-Domain Opinion Target Extraction [62.41511766918932]
  Opinion target extraction (OTE) or aspect extraction (AE) is a fundamental task in opinion mining. Recent work focus on cross-domain OTE, which is typically encountered in real-world scenarios. We propose a new SSL approach that opts for selecting target samples whose model output from a domain-specific teacher and student network disagrees on the unlabelled target data.
  arXiv  Detail & Related papers  (2023-02-28T16:31:17Z)
• FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition [42.32824906747491]
  This paper proposes a two-step rationale-centric data augmentation method to improve the model's generalization ability. Results on several datasets show that our model-agnostic method significantly improves the performance of cross-domain NER tasks.
  arXiv  Detail & Related papers  (2022-08-24T12:12:38Z)
• Focusing on Potential Named Entities During Active Label Acquisition [0.0]
  Named entity recognition (NER) aims to identify mentions of named entities in an unstructured text. Many domain-specific NER applications still call for a substantial amount of labeled data. We propose a better data-driven normalization approach to penalize sentences that are too long or too short.
  arXiv  Detail & Related papers  (2021-11-06T09:04:16Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency [93.89773386634717]
  Visual domain adaptation involves learning to classify images from a target visual domain using labels available in a different source domain. We show that in the presence of a few target labels, simple techniques like self-supervision (via rotation prediction) and consistency regularization can be effective without any adversarial alignment to learn a good target classifier. Our Pretraining and Consistency (PAC) approach, can achieve state of the art accuracy on this semi-supervised domain adaptation task, surpassing multiple adversarial domain alignment methods, across multiple datasets.
  arXiv  Detail & Related papers  (2021-01-29T18:40:17Z)
• CrossNER: Evaluating Cross-Domain Named Entity Recognition [47.9831214875796]
  Cross-domain named entity recognition models are able to cope with the scarcity issue of NER samples in target domains. Most of the existing NER benchmarks lack domain-specialized entity types or do not focus on a certain domain, leading to a less effective cross-domain evaluation. We introduce a cross-domain NER dataset (CrossNER), a fully-labeled collection of NER data spanning over five diverse domains with specialized entity categories for different domains.
  arXiv  Detail & Related papers  (2020-12-08T11:31:55Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)
• Towards Fair Cross-Domain Adaptation via Generative Learning [50.76694500782927]
  Domain Adaptation (DA) targets at adapting a model trained over the well-labeled source domain to the unlabeled target domain lying in different distributions. We develop a novel Generative Few-shot Cross-domain Adaptation (GFCA) algorithm for fair cross-domain classification.
  arXiv  Detail & Related papers  (2020-03-04T23:25:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.