Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Study

• URL: http://arxiv.org/abs/2410.20792v1
• Date: Mon, 28 Oct 2024 07:17:45 GMT
• Title: Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Study
• Authors: Jiacheng Hu, Yiru Cang, Guiran Liu, Meiqi Wang, Weijie He, Runyuan Bao,
• Abstract summary: This paper proposes a medical literature summary generation method based on the BERT model to address the challenges brought by the current explosion of medical information. We develop an efficient summary generation system that can quickly extract key information from medical literature and generate coherent, accurate summaries.
• Score: 4.416456130207115
• License:
• Abstract: This paper proposes a medical literature summary generation method based on the BERT model to address the challenges brought by the current explosion of medical information. By fine-tuning and optimizing the BERT model, we develop an efficient summary generation system that can quickly extract key information from medical literature and generate coherent, accurate summaries. In the experiment, we compared various models, including Seq-Seq, Attention, Transformer, and BERT, and demonstrated that the improved BERT model offers significant advantages in the Rouge and Recall metrics. Furthermore, the results of this study highlight the potential of knowledge distillation techniques to further enhance model performance. The system has demonstrated strong versatility and efficiency in practical applications, offering a reliable tool for the rapid screening and analysis of medical literature.

Related papers

• NeuroSym-BioCAT: Leveraging Neuro-Symbolic Methods for Biomedical Scholarly Document Categorization and Question Answering [0.14999444543328289]
  We introduce a novel approach that integrates an optimized topic modelling framework, OVB-LDA, with the BI-POP CMA-ES optimization technique for enhanced scholarly document abstract categorization. We employ the distilled MiniLM model, fine-tuned on domain-specific data, for high-precision answer extraction.
  arXiv  Detail & Related papers  (2024-10-29T14:45:12Z)
• Improving Extraction of Clinical Event Contextual Properties from Electronic Health Records: A Comparative Study [2.0884301753594334]
  This study performs a comparative analysis of various natural language models for medical text classification. BERT outperforms Bi-LSTM models by up to 28% and the baseline BERT model by up to 16% for recall of the minority classes.
  arXiv  Detail & Related papers  (2024-08-30T10:28:49Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• KG-MTT-BERT: Knowledge Graph Enhanced BERT for Multi-Type Medical Text Classification [18.99614997293521]
  KG-MTT-BERT (Knowledge Graph Enhanced Multi-Type Text BERT) is developed to deal with the complexity of medical text. Our model can outperform all baselines and other state-of-the-art models in diagnosis-related group (DRG) classification.
  arXiv  Detail & Related papers  (2022-10-08T08:37:44Z)
• SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images [62.60956024215873]
  Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. This study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification.
  arXiv  Detail & Related papers  (2022-03-22T06:54:29Z)
• BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines [49.75878234192369]
  We present WEAVER, a simple, yet efficient post-processing method that infuses old knowledge into the new model. We show that applying WEAVER in a sequential manner results in similar word embedding distributions as doing a combined training on all data at once.
  arXiv  Detail & Related papers  (2022-02-21T10:34:41Z)
• Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing [55.52858954615655]
  We conduct a systematic study on fine-tuning stability in biomedical NLP. We show that finetuning performance may be sensitive to pretraining settings, especially in low-resource domains. We show that these techniques can substantially improve fine-tuning performance for lowresource biomedical NLP applications.
  arXiv  Detail & Related papers  (2021-12-15T04:20:35Z)
• Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism [2.8881198461098894]
  We will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model.
  arXiv  Detail & Related papers  (2020-11-01T01:47:16Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)
• Literature Triage on Genomic Variation Publications by Knowledge-enhanced Multi-channel CNN [5.187865216685969]
  The aim of this study is to investigate the correlation between genomic variation and certain diseases or phenotypes. We adopt a multi-channel convolutional network to utilize rich textual information and bridge the semantic gaps from different corpora. Our model improves the accuracy of biomedical literature triage results.
  arXiv  Detail & Related papers  (2020-05-08T13:47:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.