Domain specific BERT representation for Named Entity Recognition of lab protocol

• URL: http://arxiv.org/abs/2012.11145v1
• Date: Mon, 21 Dec 2020 06:54:38 GMT
• Title: Domain specific BERT representation for Named Entity Recognition of lab protocol
• Authors: Tejas Vaidhya and Ayush Kaushal
• Abstract summary: BERT family seems to work exceptionally well on the downstream task from NER tagging to the range of other linguistic tasks. In this paper, we are going to illustrate the System for Named Entity Tagging based on Bio-Bert.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Supervised models trained to predict properties from representations have been achieving high accuracy on a variety of tasks. For instance, the BERT family seems to work exceptionally well on the downstream task from NER tagging to the range of other linguistic tasks. But the vocabulary used in the medical field contains a lot of different tokens used only in the medical industry such as the name of different diseases, devices, organisms, medicines, etc. that makes it difficult for traditional BERT model to create contextualized embedding. In this paper, we are going to illustrate the System for Named Entity Tagging based on Bio-Bert. Experimental results show that our model gives substantial improvements over the baseline and stood the fourth runner up in terms of F1 score, and first runner up in terms of Recall with just 2.21 F1 score behind the best one.

Related papers

• Multicultural Name Recognition For Previously Unseen Names [65.268245109828]
  This paper attempts to improve recognition of person names, a diverse category that can grow any time someone is born or changes their name. I look at names from 103 countries to compare how well the model performs on names from different cultures. I find that a model with combined character and word input outperforms word-only models and may improve on accuracy compared to classical NER models.
  arXiv  Detail & Related papers  (2024-01-23T17:58:38Z)
• BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining [140.61707108174247]
  We propose BioGPT, a domain-specific generative Transformer language model pre-trained on large scale biomedical literature. We get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks respectively, and 78.2% accuracy on PubMedQA.
  arXiv  Detail & Related papers  (2022-10-19T07:17:39Z)
• ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD [0.0]
  This paper presents our work to fine-tune BERT models for Arabic Word Sense Disambiguation (WSD) We constructed a dataset of labeled Arabic context-gloss pairs. Each pair was labeled as True or False and target words in each context were identified and annotated.
  arXiv  Detail & Related papers  (2022-05-19T16:47:18Z)
• Wiki to Automotive: Understanding the Distribution Shift and its impact on Named Entity Recognition [0.0]
  Transfer learning is often unable to replicate the performance of pre-trained models on text of niche domains like Automotive. We focus on performing the Named Entity Recognition (NER) task as it requires strong lexical, syntactic and semantic understanding by the model. Fine-tuning the language models with automotive domain text did not make significant improvements to the NER performance.
  arXiv  Detail & Related papers  (2021-12-01T05:13:47Z)
• Fast and Effective Biomedical Entity Linking Using a Dual Encoder [48.86736921025866]
  We propose a BERT-based dual encoder model that resolves multiple mentions in a document in one shot. We show that our proposed model is multiple times faster than existing BERT-based models while being competitive in accuracy for biomedical entity linking.
  arXiv  Detail & Related papers  (2021-03-08T19:32:28Z)
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing [73.37262264915739]
  We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
  arXiv  Detail & Related papers  (2020-07-31T00:04:15Z)
• BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision [49.42215511723874]
  We propose a new computational framework -- BOND -- to improve the prediction performance of NER models. Specifically, we propose a two-stage training algorithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels. In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance.
  arXiv  Detail & Related papers  (2020-06-28T04:55:39Z)
• Pre-training technique to localize medical BERT and enhance biomedical BERT [0.0]
  It is difficult to train specific BERT models that perform well for domains in which there are few publicly available databases of high quality and large size. We propose a single intervention with one option: simultaneous pre-training after up-sampling and amplified vocabulary. Our Japanese medical BERT outperformed conventional baselines and the other BERT models in terms of the medical document classification task.
  arXiv  Detail & Related papers  (2020-05-14T18:00:01Z)
• BURT: BERT-inspired Universal Representation from Twin Structure [89.82415322763475]
  BURT (BERT inspired Universal Representation from Twin Structure) is capable of generating universal, fixed-size representations for input sequences of any granularity. Our proposed BURT adopts the Siamese network, learning sentence-level representations from natural language inference dataset and word/phrase-level representations from paraphrasing dataset. We evaluate BURT across different granularities of text similarity tasks, including STS tasks, SemEval2013 Task 5(a) and some commonly used word similarity tasks.
  arXiv  Detail & Related papers  (2020-04-29T04:01:52Z)
• On Adversarial Examples for Biomedical NLP Tasks [4.7677261488999205]
  We propose an adversarial evaluation scheme on two well-known datasets for medical NER and STS. We show that we can significantly improve the robustness of the models by training them with adversarial examples.
  arXiv  Detail & Related papers  (2020-04-23T13:46:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.