A Lightweight Neural Model for Biomedical Entity Linking

• URL: http://arxiv.org/abs/2012.08844v1
• Date: Wed, 16 Dec 2020 10:34:37 GMT
• Title: A Lightweight Neural Model for Biomedical Entity Linking
• Authors: Lihu Chen, Ga\"el Varoquaux, Fabian M. Suchanek
• Abstract summary: We propose a lightweight neural method for biomedical entity linking. Our method uses a simple alignment layer with attention mechanisms to capture the variations between mention and entity names. Our model is competitive with previous work on standard evaluation benchmarks.
• Score: 1.8047694351309205
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biomedical entity linking aims to map biomedical mentions, such as diseases and drugs, to standard entities in a given knowledge base. The specific challenge in this context is that the same biomedical entity can have a wide range of names, including synonyms, morphological variations, and names with different word orderings. Recently, BERT-based methods have advanced the state-of-the-art by allowing for rich representations of word sequences. However, they often have hundreds of millions of parameters and require heavy computing resources, which limits their applications in resource-limited scenarios. Here, we propose a lightweight neural method for biomedical entity linking, which needs just a fraction of the parameters of a BERT model and much less computing resources. Our method uses a simple alignment layer with attention mechanisms to capture the variations between mention and entity names. Yet, we show that our model is competitive with previous work on standard evaluation benchmarks.

Related papers

• Biomedical Entity Linking as Multiple Choice Question Answering [48.74212158495695]
  We present BioELQA, a novel model that treats Biomedical Entity Linking as Multiple Choice Question Answering. We first obtains candidate entities with a fast retriever, jointly presents the mention and candidate entities to a generator, and then outputs the predicted symbol associated with its chosen entity. To improve generalization for long-tailed entities, we retrieve similar labeled training instances as clues and the input with retrieved instances for the generator.
  arXiv  Detail & Related papers  (2024-02-23T08:40:38Z)
• Multi-level biomedical NER through multi-granularity embeddings and enhanced labeling [3.8599767910528917]
  This paper proposes a hybrid approach that integrates the strengths of multiple models. BERT provides contextualized word embeddings, a pre-trained multi-channel CNN for character-level information capture, and following by a BiLSTM + CRF for sequence labelling and modelling dependencies between the words in the text. We evaluate our model on the benchmark i2b2/2010 dataset, achieving an F1-score of 90.11.
  arXiv  Detail & Related papers  (2023-12-24T21:45:36Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• Bi-Encoders based Species Normalization -- Pairwise Sentence Learning to Rank [0.0]
  We present a novel deep learning approach for named entity normalization, treating it as a pair-wise learning to rank problem. We conduct experiments on species entity types and evaluate our method against state-of-the-art techniques.
  arXiv  Detail & Related papers  (2023-10-22T17:30:16Z)
• Biomedical Language Models are Robust to Sub-optimal Tokenization [30.175714262031253]
  Most modern biomedical language models (LMs) are pre-trained using standard domain-specific tokenizers. We find that pre-training a biomedical LM using a more accurate biomedical tokenizer does not improve the entity representation quality of a language model.
  arXiv  Detail & Related papers  (2023-06-30T13:35:24Z)
• Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization [51.89486520806639]
  We propose a novel Synonym Generalization (SynGen) framework that recognizes the biomedical concepts contained in the input text using span-based predictions. We extensively evaluate our approach on a wide range of benchmarks and the results verify that SynGen outperforms previous dictionary-based models by notable margins.
  arXiv  Detail & Related papers  (2023-05-22T14:36:32Z)
• Knowledge-Rich Self-Supervised Entity Linking [58.838404666183656]
  Knowledge-RIch Self-Supervision ($tt KRISSBERT$) is a universal entity linker for four million UMLS entities. Our approach subsumes zero-shot and few-shot methods, and can easily incorporate entity descriptions and gold mention labels if available. Without using any labeled information, our method produces $tt KRISSBERT$, a universal entity linker for four million UMLS entities.
  arXiv  Detail & Related papers  (2021-12-15T05:05:12Z)
• Biomedical Entity Linking with Contrastive Context Matching [5.2710726359379265]
  We introduce BioCoM, a contrastive learning framework for biomedical entity linking. We build the training instances from raw PubMed articles by dictionary matching. We predict the normalized biomedical entity at inference time through a nearest-neighbor search.
  arXiv  Detail & Related papers  (2021-06-14T16:43:33Z)
• Fast and Effective Biomedical Entity Linking Using a Dual Encoder [48.86736921025866]
  We propose a BERT-based dual encoder model that resolves multiple mentions in a document in one shot. We show that our proposed model is multiple times faster than existing BERT-based models while being competitive in accuracy for biomedical entity linking.
  arXiv  Detail & Related papers  (2021-03-08T19:32:28Z)
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing [73.37262264915739]
  We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
  arXiv  Detail & Related papers  (2020-07-31T00:04:15Z)
• Biomedical Entity Representations with Synonym Marginalization [23.051019207472027]
  We focus on learning representations of biomedical entities solely based on the synonyms of entities. Our model-based candidates are iteratively updated to contain more difficult negative samples as our model evolves. Our model BioSyn consistently outperforms previous state-of-the-art models almost reaching the upper bound on each dataset.
  arXiv  Detail & Related papers  (2020-05-01T06:20:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.