Biomedical Entity Linking for Dutch: Fine-tuning a Self-alignment BERT Model on an Automatically Generated Wikipedia Corpus

• URL: http://arxiv.org/abs/2405.11941v1
• Date: Mon, 20 May 2024 10:30:36 GMT
• Title: Biomedical Entity Linking for Dutch: Fine-tuning a Self-alignment BERT Model on an Automatically Generated Wikipedia Corpus
• Authors: Fons Hartendorp, Tom Seinen, Erik van Mulligen, Suzan Verberne,
• Abstract summary: This paper presents the first evaluated biomedical entity linking model for the Dutch language. We derive a corpus from Wikipedia of ontology-linked Dutch biomedical entities in context. Our results indicate that biomedical entity linking in a language other than English remains challenging.
• Score: 2.4686585810894477
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biomedical entity linking, a main component in automatic information extraction from health-related texts, plays a pivotal role in connecting textual entities (such as diseases, drugs and body parts mentioned by patients) to their corresponding concepts in a structured biomedical knowledge base. The task remains challenging despite recent developments in natural language processing. This paper presents the first evaluated biomedical entity linking model for the Dutch language. We use MedRoBERTa.nl as base model and perform second-phase pretraining through self-alignment on a Dutch biomedical ontology extracted from the UMLS and Dutch SNOMED. We derive a corpus from Wikipedia of ontology-linked Dutch biomedical entities in context and fine-tune our model on this dataset. We evaluate our model on the Dutch portion of the Mantra GSC-corpus and achieve 54.7% classification accuracy and 69.8% 1-distance accuracy. We then perform a case study on a collection of unlabeled, patient-support forum data and show that our model is hampered by the limited quality of the preceding entity recognition step. Manual evaluation of small sample indicates that of the correctly extracted entities, around 65% is linked to the correct concept in the ontology. Our results indicate that biomedical entity linking in a language other than English remains challenging, but our Dutch model can be used to for high-level analysis of patient-generated text.

Related papers

• Integrating curation into scientific publishing to train AI models [1.6982459897303823]
  We have embedded multimodal data curation into the academic publishing process to annotate segmented figure panels and captions. The dataset, SourceData-NLP, contains more than 620,000 annotated biomedical entities. We evaluate the utility of the dataset to train AI models using named-entity recognition, segmentation of figure captions into their constituent panels, and a novel context-dependent semantic task.
  arXiv  Detail & Related papers  (2023-10-31T13:22:38Z)
• Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking [4.8882241537236455]
  This research investigates a method that exploits the in-context learning capabilities of large models for biomedical concept linking. The proposed approach adopts a two-stage retrieve-and-rank framework. It achieved an accuracy of 90.% in BC5CDR disease entity normalization and 94.7% in chemical entity normalization.
  arXiv  Detail & Related papers  (2023-07-03T16:19:50Z)
• Biomedical Language Models are Robust to Sub-optimal Tokenization [30.175714262031253]
  Most modern biomedical language models (LMs) are pre-trained using standard domain-specific tokenizers. We find that pre-training a biomedical LM using a more accurate biomedical tokenizer does not improve the entity representation quality of a language model.
  arXiv  Detail & Related papers  (2023-06-30T13:35:24Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• RuBioRoBERTa: a pre-trained biomedical language model for Russian language biomedical text mining [117.56261821197741]
  We present several BERT-based models for Russian language biomedical text mining. The models are pre-trained on a corpus of freely available texts in the Russian biomedical domain.
  arXiv  Detail & Related papers  (2022-04-08T09:18:59Z)
• Towards more patient friendly clinical notes through language models and ontologies [57.51898902864543]
  We present a novel approach to automated medical text based on word simplification and language modelling. We use a new dataset pairs of publicly available medical sentences and a version of them simplified by clinicians. Our method based on a language model trained on medical forum data generates simpler sentences while preserving both grammar and the original meaning.
  arXiv  Detail & Related papers  (2021-12-23T16:11:19Z)
• Automated tabulation of clinical trial results: A joint entity and relation extraction approach with transformer-based language representations [5.825190876052148]
  This paper investigates automating evidence table generation by decomposing the problem across two language processing tasks. We focus on the automatic tabulation of sentences from published RCT abstracts that report the practice outcomes. To train and test these models, a new gold-standard corpus was developed, comprising almost 600 result sentences from six disease areas.
  arXiv  Detail & Related papers  (2021-12-10T15:26:43Z)
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark [51.38557174322772]
  We present the first Chinese Biomedical Language Understanding Evaluation benchmark. It is a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification. We report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling.
  arXiv  Detail & Related papers  (2021-06-15T12:25:30Z)
• An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets [94.37521840642141]
  We present the full-size Russian complexly NER-labeled corpus of Internet user reviews. A set of advanced deep learning neural networks is used to extract pharmacologically meaningful entities from Russian texts.
  arXiv  Detail & Related papers  (2021-04-30T19:46:24Z)
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing [73.37262264915739]
  We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
  arXiv  Detail & Related papers  (2020-07-31T00:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.