How Do Your Biomedical Named Entity Models Generalize to Novel Entities?

• URL: http://arxiv.org/abs/2101.00160v1
• Date: Fri, 1 Jan 2021 04:13:42 GMT
• Title: How Do Your Biomedical Named Entity Models Generalize to Novel Entities?
• Authors: Hyunjae Kim, Jaewoo Kang
• Abstract summary: We analyze the three types of recognition abilities of BioNER models: memorization, synonym generalization, and concept generalization. We find that (1) BioNER models are overestimated in terms of their generalization ability, and (2) they tend to exploit dataset biases, which hinders the models' abilities to generalize. Our method consistently improves the generalizability of the state-of-the-art (SOTA) models on five benchmark datasets, allowing them to better perform on unseen entity mentions.
• Score: 17.83980569600546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The number of biomedical literature on new biomedical concepts is rapidly increasing, which necessitates a reliable biomedical named entity recognition (BioNER) model for identifying new and unseen entity mentions. However, it is questionable whether existing BioNER models can effectively handle them. In this work, we systematically analyze the three types of recognition abilities of BioNER models: memorization, synonym generalization, and concept generalization. We find that (1) BioNER models are overestimated in terms of their generalization ability, and (2) they tend to exploit dataset biases, which hinders the models' abilities to generalize. To enhance the generalizability, we present a simple debiasing method based on the data statistics. Our method consistently improves the generalizability of the state-of-the-art (SOTA) models on five benchmark datasets, allowing them to better perform on unseen entity mentions.

Related papers

• Augmenting Biomedical Named Entity Recognition with General-domain Resources [47.24727904076347]
  Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. We propose GERBERA, a simple-yet-effective method that utilized a general-domain NER dataset for training. We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances.
  arXiv  Detail & Related papers  (2024-06-15T15:28:02Z)
• BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments [112.25067497985447]
  We introduce BioDiscoveryAgent, an agent that designs new experiments, reasons about their outcomes, and efficiently navigates the hypothesis space to reach desired solutions. BioDiscoveryAgent can uniquely design new experiments without the need to train a machine learning model. It achieves an average of 21% improvement in predicting relevant genetic perturbations across six datasets.
  arXiv  Detail & Related papers  (2024-05-27T19:57:17Z)
• Multi-level biomedical NER through multi-granularity embeddings and enhanced labeling [3.8599767910528917]
  This paper proposes a hybrid approach that integrates the strengths of multiple models. BERT provides contextualized word embeddings, a pre-trained multi-channel CNN for character-level information capture, and following by a BiLSTM + CRF for sequence labelling and modelling dependencies between the words in the text. We evaluate our model on the benchmark i2b2/2010 dataset, achieving an F1-score of 90.11.
  arXiv  Detail & Related papers  (2023-12-24T21:45:36Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks [68.39821375903591]
  Generalist AI holds the potential to address limitations due to its versatility in interpreting different data types. Here, we propose BiomedGPT, the first open-source and lightweight vision-language foundation model.
  arXiv  Detail & Related papers  (2023-05-26T17:14:43Z)
• BioAug: Conditional Generation based Data Augmentation for Low-Resource Biomedical NER [52.79573512427998]
  We present BioAug, a novel data augmentation framework for low-resource BioNER. BioAug is trained to solve a novel text reconstruction task based on selective masking and knowledge augmentation. We demonstrate the effectiveness of BioAug on 5 benchmark BioNER datasets.
  arXiv  Detail & Related papers  (2023-05-18T02:04:38Z)
• Recognising Biomedical Names: Challenges and Solutions [9.51284672475743]
  We propose a transition-based NER model which can recognise discontinuous mentions. We also develop a cost-effective approach that nominates the suitable pre-training data. Our contributions have obvious practical implications, especially when new biomedical applications are needed.
  arXiv  Detail & Related papers  (2021-06-23T08:20:13Z)
• Biomedical Interpretable Entity Representations [40.6095537182194]
  Pre-trained language models induce dense entity representations that offer strong performance on entity-centric NLP tasks. This can be a barrier to model uptake in important domains such as biomedicine. We create a new entity type system and training set from a large corpus of biomedical texts.
  arXiv  Detail & Related papers  (2021-06-17T13:52:10Z)
• BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition [9.05154470433578]
  Existing BioNER approaches often neglect these issues and directly adopt the state-of-the-art (SOTA) models. We propose biomedical ALBERT, an effective domain-specific language model trained on large-scale biomedical corpora.
  arXiv  Detail & Related papers  (2020-09-19T12:58:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.