Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition

• URL: http://arxiv.org/abs/2011.00425v1
• Date: Sun, 1 Nov 2020 04:52:56 GMT
• Title: Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition
• Authors: Arda Akdemir and Tetsuo Shibuya
• Abstract summary: State-of-the-art deep-learning based solutions for entity recognition often require large annotated datasets. We performed an extensive analysis to understand the transferability between different biomedical entity datasets. We propose combining transfer learning and multi-task learning to improve the performance of biomedical named entity recognition systems.
• Score: 6.09170287691728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Developing high-performing systems for detecting biomedical named entities has major implications. State-of-the-art deep-learning based solutions for entity recognition often require large annotated datasets, which is not available in the biomedical domain. Transfer learning and multi-task learning have been shown to improve performance for low-resource domains. However, the applications of these methods are relatively scarce in the biomedical domain, and a theoretical understanding of why these methods improve the performance is lacking. In this study, we performed an extensive analysis to understand the transferability between different biomedical entity datasets. We found useful measures to predict transferability between these datasets. Besides, we propose combining transfer learning and multi-task learning to improve the performance of biomedical named entity recognition systems, which is not applied before to the best of our knowledge.

Related papers

• Causal Representation Learning from Multimodal Biological Observations [57.00712157758845]
  We aim to develop flexible identification conditions for multimodal data. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• BioMNER: A Dataset for Biomedical Method Entity Recognition [25.403593761614424]
  We propose a novel dataset for biomedical method entity recognition. We employ an automated BioMethod entity recognition and information retrieval system to assist human annotation. Our empirical findings reveal that the large parameter counts of language models surprisingly inhibit the effective assimilation of entity extraction patterns.
  arXiv  Detail & Related papers  (2024-06-28T16:34:24Z)
• Knowledge-guided EEG Representation Learning [27.8095014391814]
  Self-supervised learning has produced impressive results in multimedia domains of audio, vision and speech. We propose a self-supervised model for EEG, which provides robust performance and remarkable parameter efficiency. We also propose a novel knowledge-guided pre-training objective that accounts for the idiosyncrasies of the EEG signal.
  arXiv  Detail & Related papers  (2024-02-15T01:52:44Z)
• Multi-level biomedical NER through multi-granularity embeddings and enhanced labeling [3.8599767910528917]
  This paper proposes a hybrid approach that integrates the strengths of multiple models. BERT provides contextualized word embeddings, a pre-trained multi-channel CNN for character-level information capture, and following by a BiLSTM + CRF for sequence labelling and modelling dependencies between the words in the text. We evaluate our model on the benchmark i2b2/2010 dataset, achieving an F1-score of 90.11.
  arXiv  Detail & Related papers  (2023-12-24T21:45:36Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab [67.24684071577211]
  The challenge of replicating research results has posed a significant impediment to the field of molecular biology. We first curate a comprehensive multimodal dataset, named ProBio, as an initial step towards this objective. Next, we devise two challenging benchmarks, transparent solution tracking and multimodal action recognition, to emphasize the unique characteristics and difficulties associated with activity understanding in BioLab settings.
  arXiv  Detail & Related papers  (2023-11-01T14:44:01Z)
• Machine learning in bioprocess development: From promise to practice [58.720142291102135]
  Data-driven methods like machine learning (ML) approaches have a high potential to rationally explore large design spaces. The aim of this review is to demonstrate how ML methods have been applied so far in bioprocess development.
  arXiv  Detail & Related papers  (2022-10-04T13:48:59Z)
• BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Task [4.837365865245979]
  We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task. BioADAPT-MRC is a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets.
  arXiv  Detail & Related papers  (2022-02-26T16:14:27Z)
• Discovering Drug-Target Interaction Knowledge from Biomedical Literature [107.98712673387031]
  The Interaction between Drugs and Targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedical domain, automatically discovering DTI knowledge from literature becomes an urgent demand in the industry. We explore the first end-to-end solution for this task by using generative approaches. We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations.
  arXiv  Detail & Related papers  (2021-09-27T17:00:14Z)
• Network Module Detection from Multi-Modal Node Features with a Greedy Decision Forest for Actionable Explainable AI [0.0]
  In this work, we demonstrate subnetwork detection based on multi-modal node features using a new Greedy Decision Forest. Our glass-box approach could help to uncover disease-causing network modules from multi-omics data to better understand diseases such as cancer.
  arXiv  Detail & Related papers  (2021-08-26T09:42:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.