Biomedical Entity Linking with Contrastive Context Matching

• URL: http://arxiv.org/abs/2106.07583v2
• Date: Tue, 15 Jun 2021 05:47:29 GMT
• Title: Biomedical Entity Linking with Contrastive Context Matching
• Authors: Shogo Ujiie, Hayate Iso, Eiji Aramaki
• Abstract summary: We introduce BioCoM, a contrastive learning framework for biomedical entity linking. We build the training instances from raw PubMed articles by dictionary matching. We predict the normalized biomedical entity at inference time through a nearest-neighbor search.
• Score: 5.2710726359379265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce BioCoM, a contrastive learning framework for biomedical entity linking that uses only two resources: a small-sized dictionary and a large number of raw biomedical articles. Specifically, we build the training instances from raw PubMed articles by dictionary matching and use them to train a context-aware entity linking model with contrastive learning. We predict the normalized biomedical entity at inference time through a nearest-neighbor search. Results found that BioCoM substantially outperforms state-of-the-art models, especially in low-resource settings, by effectively using the context of the entities.

Related papers

• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• High-throughput Biomedical Relation Extraction for Semi-Structured Web Articles Empowered by Large Language Models [1.9665865095034865]
  We formulate the relation extraction task as binary classifications for large language models. We designate the main title as the tail entity and explicitly incorporate it into the context. Longer contents are sliced into text chunks, embedded, and retrieved with additional embedding models.
  arXiv  Detail & Related papers  (2023-12-13T16:43:41Z)
• Biomedical Entity Linking with Triple-aware Pre-Training [7.536753993136013]
  We propose a framework to pre-train a powerful large language model (LLM) by a corpus synthesized from a KG. In the evaluations we are unable to confirm the benefit of including synonym, description or relational information.
  arXiv  Detail & Related papers  (2023-08-28T09:06:28Z)
• Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking [4.8882241537236455]
  This research investigates a method that exploits the in-context learning capabilities of large models for biomedical concept linking. The proposed approach adopts a two-stage retrieve-and-rank framework. It achieved an accuracy of 90.% in BC5CDR disease entity normalization and 94.7% in chemical entity normalization.
  arXiv  Detail & Related papers  (2023-07-03T16:19:50Z)
• Biomedical Language Models are Robust to Sub-optimal Tokenization [30.175714262031253]
  Most modern biomedical language models (LMs) are pre-trained using standard domain-specific tokenizers. We find that pre-training a biomedical LM using a more accurate biomedical tokenizer does not improve the entity representation quality of a language model.
  arXiv  Detail & Related papers  (2023-06-30T13:35:24Z)
• EBOCA: Evidences for BiOmedical Concepts Association Ontology [55.41644538483948]
  This paper proposes EBOCA, an ontology that describes (i) biomedical domain concepts and associations between them, and (ii) evidences supporting these associations. Test data coming from a subset of DISNET and automatic association extractions from texts has been transformed to create a Knowledge Graph that can be used in real scenarios.
  arXiv  Detail & Related papers  (2022-08-01T18:47:03Z)
• Discovering Drug-Target Interaction Knowledge from Biomedical Literature [107.98712673387031]
  The Interaction between Drugs and Targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedical domain, automatically discovering DTI knowledge from literature becomes an urgent demand in the industry. We explore the first end-to-end solution for this task by using generative approaches. We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations.
  arXiv  Detail & Related papers  (2021-09-27T17:00:14Z)
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark [51.38557174322772]
  We present the first Chinese Biomedical Language Understanding Evaluation benchmark. It is a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification. We report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling.
  arXiv  Detail & Related papers  (2021-06-15T12:25:30Z)
• Fast and Effective Biomedical Entity Linking Using a Dual Encoder [48.86736921025866]
  We propose a BERT-based dual encoder model that resolves multiple mentions in a document in one shot. We show that our proposed model is multiple times faster than existing BERT-based models while being competitive in accuracy for biomedical entity linking.
  arXiv  Detail & Related papers  (2021-03-08T19:32:28Z)
• A Lightweight Neural Model for Biomedical Entity Linking [1.8047694351309205]
  We propose a lightweight neural method for biomedical entity linking. Our method uses a simple alignment layer with attention mechanisms to capture the variations between mention and entity names. Our model is competitive with previous work on standard evaluation benchmarks.
  arXiv  Detail & Related papers  (2020-12-16T10:34:37Z)
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing [73.37262264915739]
  We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
  arXiv  Detail & Related papers  (2020-07-31T00:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.