AUC Maximization for Low-Resource Named Entity Recognition

• URL: http://arxiv.org/abs/2212.04800v3
• Date: Thu, 13 Apr 2023 18:52:56 GMT
• Title: AUC Maximization for Low-Resource Named Entity Recognition
• Abstract summary: Current work in named entity recognition (NER) uses either cross entropy (CE) or conditional random fields (CRF) as the objective/loss functions. We propose to optimize the NER model by maximizing the AUC score.
• Score: 37.31342703194106
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current work in named entity recognition (NER) uses either cross entropy (CE) or conditional random fields (CRF) as the objective/loss functions to optimize the underlying NER model. Both of these traditional objective functions for the NER problem generally produce adequate performance when the data distribution is balanced and there are sufficient annotated training examples. But since NER is inherently an imbalanced tagging problem, the model performance under the low-resource settings could suffer using these standard objective functions. Based on recent advances in area under the ROC curve (AUC) maximization, we propose to optimize the NER model by maximizing the AUC score. We give evidence that by simply combining two binary-classifiers that maximize the AUC score, significant performance improvement over traditional loss functions is achieved under low-resource NER settings. We also conduct extensive experiments to demonstrate the advantages of our method under the low-resource and highly-imbalanced data distribution settings. To the best of our knowledge, this is the first work that brings AUC maximization to the NER setting. Furthermore, we show that our method is agnostic to different types of NER embeddings, models and domains. The code to replicate this work will be provided upon request.

Related papers

• NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learning [12.75939447499315]
  We propose NE-R1, a novel framework for adaptive retrieval-augmented NER.<n>We design a "retrieval-on-demand" mechanism for NER. Then we integrate it into models by a two-stage training method.<n> NE-R1 achieves state-of-the-art performance on various benchmarks, with an average F1 score gain of 2.52% in in-domain evaluation and 1.18% in zero-shot cross-domain evaluation.
  arXiv  Detail & Related papers  (2026-09-02T09:37:16Z)
• Beyond Static Best-of-N: Bayesian List-wise Alignment for LLM-based Recommendation [23.95623437107717]
  Large Language Models have revolutionized recommender systems (LLM4Rec) by leveraging their generative capabilities to model user preferences.<n>Existing LLM4Rec methods rely on token-level objectives, making it difficult to optimize list-level and non-differentiable metrics that define actual recommendation quality.<n>We propose BLADE (Bayesian List-wise Alignment via Dynamic Estimation) to overcome these challenges.
  arXiv  Detail & Related papers  (2026-05-06T07:02:57Z)
• Stochastic Primal-Dual Double Block-Coordinate for Two-way Partial AUC Maximization [56.805574957824135]
  Two-way partial AUCAUC is a critical performance metric for binary classification with imbalanced data.<n>Existing algorithms for TPAUC optimization remain under-explored.<n>We introduce two innovative double-coordinate block-coordinate algorithms for TPAUC optimization.
  arXiv  Detail & Related papers  (2025-05-28T03:55:05Z)
• Distributionally Robust Reinforcement Learning with Human Feedback [13.509499718691016]
  We introduce a distributionally robust RLHF for fine-tuning large language models. Our goal is to ensure that a fine-tuned model retains its performance even when the distribution of prompts significantly differs. We show that our robust training improves the accuracy of the learned reward models on average, and markedly on some tasks, such as reasoning.
  arXiv  Detail & Related papers  (2025-03-01T15:43:39Z)
• RECOMBINER: Robust and Enhanced Compression with Bayesian Implicit Neural Representations [8.417694229876371]
  COMBINER avoids quantization and enables direct optimization of the rate-distortion performance. We propose Robust and Enhanced COMBINER (RECOMBINER) to overcome COMBINER's limitations. We show that RECOMBINER achieves competitive results with the best INR-based methods and even outperforms autoencoder-based codecs on low-resolution images.
  arXiv  Detail & Related papers  (2023-09-29T12:27:15Z)
• Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints [51.12047280149546]
  A direct approach for obtaining a fair predictive model is to train the model through optimizing its prediction performance subject to fairness constraints. We formulate the training problem of a fairness-aware machine learning model as an AUC optimization problem subject to a class of AUC-based fairness constraints. We demonstrate the effectiveness of our approach on real-world data under different fairness metrics.
  arXiv  Detail & Related papers  (2022-12-23T22:29:08Z)
• On the Effective Usage of Priors in RSS-based Localization [56.68864078417909]
  We propose a Received Signal Strength (RSS) fingerprint and convolutional neural network-based algorithm, LocUNet. In this paper, we study the localization problem in dense urban settings. We first recognize LocUNet's ability to learn the underlying prior distribution of the Rx position or Rx and transmitter (Tx) association preferences from the training data, and attribute its high performance to these.
  arXiv  Detail & Related papers  (2022-11-28T00:31:02Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Optimizing Two-way Partial AUC with an End-to-end Framework [154.47590401735323]
  Area Under the ROC Curve (AUC) is a crucial metric for machine learning. Recent work shows that the TPAUC is essentially inconsistent with the existing Partial AUC metrics. We present the first trial in this paper to optimize this new metric.
  arXiv  Detail & Related papers  (2022-06-23T12:21:30Z)
• Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder [23.860842627883187]
  We teach the model to capture broader variations of the feature distributions with a novel noise-enhanced supervised autoencoder (NSAE) NSAE trains the model by jointly reconstructing inputs and predicting the labels of inputs as well as their reconstructed pairs. We also take advantage of NSAE structure and propose a two-step fine-tuning procedure that achieves better adaption and improves classification performance in the target domain.
  arXiv  Detail & Related papers  (2021-08-11T04:45:56Z)
• Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition [8.344476599818826]
  We introduce NRB, a new testbed designed to diagnose Name Regularity Bias of NER models. Our results indicate that all state-of-the-art models we tested show such a bias. We propose a novel model-agnostic training method that adds learnable adversarial noise to some entity mentions.
  arXiv  Detail & Related papers  (2021-07-24T13:55:35Z)
• Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence [66.83161885378192]
  Area under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. We propose a technical method to optimize AUPRC for deep learning.
  arXiv  Detail & Related papers  (2021-04-18T06:22:21Z)
• Adaptive Name Entity Recognition under Highly Unbalanced Data [5.575448433529451]
  We present our experiments on a neural architecture composed of a Conditional Random Field (CRF) layer stacked on top of a Bi-directional LSTM (BI-LSTM) layer for solving NER tasks. We introduce an add-on classification model to split sentences into two different sets: Weak and Strong classes and then designing a couple of Bi-LSTM-CRF models properly to optimize performance on each set.
  arXiv  Detail & Related papers  (2020-03-10T06:56:52Z)
• A Distributionally Robust Area Under Curve Maximization Model [1.370633147306388]
  We propose two new distributionally robust AUC models (DR-AUC) DR-AUC models rely on the Kantorovich metric and approximate the AUC with the hinge loss function. numerical experiments show that the proposed DR-AUC models perform better in general and in particular improve the worst-case out-of-sample performance.
  arXiv  Detail & Related papers  (2020-02-18T02:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.