Multi-level biomedical NER through multi-granularity embeddings and
enhanced labeling
- URL: http://arxiv.org/abs/2312.15550v1
- Date: Sun, 24 Dec 2023 21:45:36 GMT
- Title: Multi-level biomedical NER through multi-granularity embeddings and
enhanced labeling
- Authors: Fahime Shahrokh, Nasser Ghadiri, Rasoul Samani, Milad Moradi
- Abstract summary: This paper proposes a hybrid approach that integrates the strengths of multiple models.
BERT provides contextualized word embeddings, a pre-trained multi-channel CNN for character-level information capture, and following by a BiLSTM + CRF for sequence labelling and modelling dependencies between the words in the text.
We evaluate our model on the benchmark i2b2/2010 dataset, achieving an F1-score of 90.11.
- Score: 3.8599767910528917
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Biomedical Named Entity Recognition (NER) is a fundamental task of Biomedical
Natural Language Processing for extracting relevant information from biomedical
texts, such as clinical records, scientific publications, and electronic health
records. The conventional approaches for biomedical NER mainly use traditional
machine learning techniques, such as Conditional Random Fields and Support
Vector Machines or deep learning-based models like Recurrent Neural Networks
and Convolutional Neural Networks. Recently, Transformer-based models,
including BERT, have been used in the domain of biomedical NER and have
demonstrated remarkable results. However, these models are often based on
word-level embeddings, limiting their ability to capture character-level
information, which is effective in biomedical NER due to the high variability
and complexity of biomedical texts. To address these limitations, this paper
proposes a hybrid approach that integrates the strengths of multiple models. In
this paper, we proposed an approach that leverages fine-tuned BERT to provide
contextualized word embeddings, a pre-trained multi-channel CNN for
character-level information capture, and following by a BiLSTM + CRF for
sequence labelling and modelling dependencies between the words in the text. In
addition, also we propose an enhanced labelling method as part of
pre-processing to enhance the identification of the entity's beginning word and
thus improve the identification of multi-word entities, a common challenge in
biomedical NER. By integrating these models and the pre-processing method, our
proposed model effectively captures both contextual information and detailed
character-level information. We evaluated our model on the benchmark i2b2/2010
dataset, achieving an F1-score of 90.11. These results illustrate the
proficiency of our proposed model in performing biomedical Named Entity
Recognition.
Related papers
- BioMNER: A Dataset for Biomedical Method Entity Recognition [25.403593761614424]
We propose a novel dataset for biomedical method entity recognition.
We employ an automated BioMethod entity recognition and information retrieval system to assist human annotation.
Our empirical findings reveal that the large parameter counts of language models surprisingly inhibit the effective assimilation of entity extraction patterns.
arXiv Detail & Related papers (2024-06-28T16:34:24Z) - Diversifying Knowledge Enhancement of Biomedical Language Models using
Adapter Modules and Knowledge Graphs [54.223394825528665]
We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models.
We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT.
We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
arXiv Detail & Related papers (2023-12-21T14:26:57Z) - UMLS-KGI-BERT: Data-Centric Knowledge Integration in Transformers for
Biomedical Entity Recognition [4.865221751784403]
This work contributes a data-centric paradigm for enriching the language representations of biomedical transformer-encoder LMs by extracting text sequences from the UMLS.
Preliminary results from experiments in the extension of pre-trained LMs as well as training from scratch show that this framework improves downstream performance on multiple biomedical and clinical Named Entity Recognition (NER) tasks.
arXiv Detail & Related papers (2023-07-20T18:08:34Z) - BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks [68.39821375903591]
Generalist AI holds the potential to address limitations due to its versatility in interpreting different data types.
Here, we propose BiomedGPT, the first open-source and lightweight vision-language foundation model.
arXiv Detail & Related papers (2023-05-26T17:14:43Z) - BiomedCLIP: a multimodal biomedical foundation model pretrained from
fifteen million scientific image-text pairs [48.376109878173956]
We present PMC-15M, a novel dataset that is two orders of magnitude larger than existing biomedical multimodal datasets.
PMC-15M contains 15 million biomedical image-text pairs collected from 4.4 million scientific articles.
Based on PMC-15M, we have pretrained BiomedCLIP, a multimodal foundation model, with domain-specific adaptations tailored to biomedical vision-language processing.
arXiv Detail & Related papers (2023-03-02T02:20:04Z) - Fine-tuning Pretrained Language Models with Label Attention for
Explainable Biomedical Text Classification [1.066048003460524]
We develop an improved label attention-based architecture to inject semantic label description into the fine-tuning process of PTMs.
Results on two public medical datasets show that the proposed fine-tuning scheme outperforms the conventionally fine-tuned PTMs and prior state-of-the-art models.
arXiv Detail & Related papers (2021-08-26T14:23:06Z) - Recognising Biomedical Names: Challenges and Solutions [9.51284672475743]
We propose a transition-based NER model which can recognise discontinuous mentions.
We also develop a cost-effective approach that nominates the suitable pre-training data.
Our contributions have obvious practical implications, especially when new biomedical applications are needed.
arXiv Detail & Related papers (2021-06-23T08:20:13Z) - G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for
Biomarker Identification and Disease Classification [49.53651166356737]
We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers.
We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
arXiv Detail & Related papers (2021-01-27T19:28:04Z) - UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual
Embeddings Using the Unified Medical Language System Metathesaurus [73.86656026386038]
We introduce UmlsBERT, a contextual embedding model that integrates domain knowledge during the pre-training process.
By applying these two strategies, UmlsBERT can encode clinical domain knowledge into word embeddings and outperform existing domain-specific models.
arXiv Detail & Related papers (2020-10-20T15:56:31Z) - BioALBERT: A Simple and Effective Pre-trained Language Model for
Biomedical Named Entity Recognition [9.05154470433578]
Existing BioNER approaches often neglect these issues and directly adopt the state-of-the-art (SOTA) models.
We propose biomedical ALBERT, an effective domain-specific language model trained on large-scale biomedical corpora.
arXiv Detail & Related papers (2020-09-19T12:58:47Z) - Domain-Specific Language Model Pretraining for Biomedical Natural
Language Processing [73.37262264915739]
We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains.
Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
arXiv Detail & Related papers (2020-07-31T00:04:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.