Diagnosis extraction from unstructured Dutch echocardiogram reports using span- and document-level characteristic classification

• URL: http://arxiv.org/abs/2408.06930v2
• Date: Thu, 15 Aug 2024 12:42:12 GMT
• Title: Diagnosis extraction from unstructured Dutch echocardiogram reports using span- and document-level characteristic classification
• Abstract summary: This study tests the feasibility of automatic span- and document-level diagnosis extraction from unstructured Dutch echocardiogram reports. We developed and tested several automatic labelling techniques at both span and document levels, using weighted and macro F1-score, precision, and recall for performance evaluation. The SpanCategorizer and MedRoBERTa$.$nl models outperformed all other span and document classifiers, respectively.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical machine learning research and AI driven clinical decision support models rely on clinically accurate labels. Manually extracting these labels with the help of clinical specialists is often time-consuming and expensive. This study tests the feasibility of automatic span- and document-level diagnosis extraction from unstructured Dutch echocardiogram reports. We included 115,692 unstructured echocardiogram reports from the UMCU a large university hospital in the Netherlands. A randomly selected subset was manually annotated for the occurrence and severity of eleven commonly described cardiac characteristics. We developed and tested several automatic labelling techniques at both span and document levels, using weighted and macro F1-score, precision, and recall for performance evaluation. We compared the performance of span labelling against document labelling methods, which included both direct document classifiers and indirect document classifiers that rely on span classification results. The SpanCategorizer and MedRoBERTa$.$nl models outperformed all other span and document classifiers, respectively. The weighted F1-score varied between characteristics, ranging from 0.60 to 0.93 in SpanCategorizer and 0.96 to 0.98 in MedRoBERTa$.$nl. Direct document classification was superior to indirect document classification using span classifiers. SetFit achieved competitive document classification performance using only 10% of the training data. Utilizing a reduced label set yielded near-perfect document classification results. We recommend using our published SpanCategorizer and MedRoBERTa$.$nl models for span- and document-level diagnosis extraction from Dutch echocardiography reports. For settings with limited training data, SetFit may be a promising alternative for document classification.

Related papers

• Clinical Term Extraction using Open-Source Small Language Models [2.5316486726943843]
  Open-source small language models were evaluated for ALS-relevant clinical terms in patient documentation.<n>We compared 26 open-source models using aggregate, label-level, and manual-validation multilabel classification metrics.<n>These findings support targeted hybrid extraction rather than replacement of existing rule-based methods.
  arXiv  Detail & Related papers  (2026-06-19T18:59:33Z)
• CLiGNet: Clinical Label-Interaction Graph Network for Medical Specialty Classification from Clinical Transcriptions [0.0]
  We first document this methodological flaw and establish a leakage free benchmark across 40 medical specialties (4966 records)<n>We then introduce CLiGNet, a neural architecture that combines a Bio ClinicalBERT text encoder with a two layer Graph Convolutional Network operating on a specialty label graph constructed from semantic similarity and ICD 10 chapter priors.<n>We provide comprehensive failure analysis covering pairwise specialty confusions, rare class behaviour, document length effects, and token level Integrated Gradients attribution, offering actionable insights for clinical NLP system deployment.
  arXiv  Detail & Related papers  (2026-03-24T03:30:06Z)
• CNSight: Evaluation of Clinical Note Segmentation Tools [3.673249612734457]
  We evaluate rule-based baselines, domain-specific transformer models, and large language models for clinical note segmentation using a curated dataset of 1,000 notes from MIMIC-IV.<n>Our experiments show that large API-based models achieve the best overall performance, with GPT-5-mini reaching a best average F1 of 72.4 across sentence-level and freetext segmentation.
  arXiv  Detail & Related papers  (2025-12-28T05:40:15Z)
• Using LLMs for Multilingual Clinical Entity Linking to ICD-10 [3.7463543521744764]
  We propose an approach for linking clinical terms to ICD-10 codes in different languages using Large Language Models (LLMs)<n>Our system shows promising results in predicting ICD-10 codes on different benchmark datasets in Spanish.
  arXiv  Detail & Related papers  (2025-09-05T07:30:40Z)
• Blueprinting the Future: Automatic Item Categorization using Hierarchical Zero-Shot and Few-Shot Classifiers [6.907552533477328]
  This study unveils a novel approach employing the zero-shot and few-shot Generative Pretrained Transformer (GPT) for hierarchical item categorization. The hierarchical nature of examination blueprints is navigated seamlessly, allowing for a tiered classification of items across multiple levels. An initial simulation with artificial data demonstrates the efficacy of this method, achieving an average accuracy of 92.91% measured by the F1 score.
  arXiv  Detail & Related papers  (2023-12-06T15:51:49Z)
• Detecting automatically the layout of clinical documents to enhance the performances of downstream natural language processing [53.797797404164946]
  We designed an algorithm to process clinical PDF documents and extract only clinically relevant text. The algorithm consists of several steps: initial text extraction using a PDF, followed by classification into such categories as body text, left notes, and footers. Medical performance was evaluated by examining the extraction of medical concepts of interest from the text in their respective sections.
  arXiv  Detail & Related papers  (2023-05-23T08:38:33Z)
• Development and validation of a natural language processing algorithm to pseudonymize documents in the context of a clinical data warehouse [53.797797404164946]
  The study highlights the difficulties faced in sharing tools and resources in this domain. We annotated a corpus of clinical documents according to 12 types of identifying entities. We build a hybrid system, merging the results of a deep learning model as well as manual rules.
  arXiv  Detail & Related papers  (2023-03-23T17:17:46Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• Statistical Dependency Guided Contrastive Learning for Multiple Labeling in Prenatal Ultrasound [56.631021151764955]
  Standard plane recognition plays an important role in prenatal ultrasound (US) screening. We build a novel multi-label learning scheme to identify multiple standard planes and corresponding anatomical structures simultaneously.
  arXiv  Detail & Related papers  (2021-08-11T06:39:26Z)
• Detecting of a Patient's Condition From Clinical Narratives Using Natural Language Representation [0.3149883354098941]
  This paper proposes a joint clinical natural language representation learning and supervised classification framework. The novel framework jointly discovers distributional syntactic and latent semantic (representation learning) from contextual clinical narrative inputs. The proposed framework yields an overall classification performance with accuracy, recall, and precision of 89 % and 88 %, 89 %, respectively.
  arXiv  Detail & Related papers  (2021-04-08T17:16:04Z)
• SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models [48.374678491735665]
  We introduce Set-classifier with Class-specific RIsk Bounds (SCRIB) to tackle this problem. SCRIB constructs a set-classifier that controls the class-specific prediction risks with a theoretical guarantee. We validated SCRIB on several medical applications, including sleep staging on electroencephalogram (EEG) data, X-ray COVID image classification, and atrial fibrillation detection based on electrocardiogram (ECG) data.
  arXiv  Detail & Related papers  (2021-03-05T21:06:12Z)
• Medical Segment Coloring of Clinical Notes [0.0]
  This paper proposes a deep learning-based method to identify the segments of a clinical note corresponding to ICD-9 broad categories. The proposed Medical Segment Colorer (MSC) architecture is a pipeline framework that works in three stages: word categorization, (2) phrase allocation, and (3) document classification.
  arXiv  Detail & Related papers  (2021-01-26T09:49:37Z)
• Unsupervised Label Refinement Improves Dataless Text Classification [48.031421660674745]
  Dataless text classification is capable of classifying documents into previously unseen labels by assigning a score to any document paired with a label description. While promising, it crucially relies on accurate descriptions of the label set for each downstream task. This reliance causes dataless classifiers to be highly sensitive to the choice of label descriptions and hinders the broader application of dataless classification in practice.
  arXiv  Detail & Related papers  (2020-12-08T03:37:50Z)
• Learning Image Labels On-the-fly for Training Robust Classification Models [13.669654965671604]
  We show how noisy annotations (e.g., from different algorithm-based labelers) can be utilized together and mutually benefit the learning of classification tasks. A meta-training based label-sampling module is designed to attend the labels that benefit the model learning the most through additional back-propagation processes.
  arXiv  Detail & Related papers  (2020-09-22T05:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.