Clinical Term Extraction using Open-Source Small Language Models

• URL: http://arxiv.org/abs/2606.21689v1
• Date: Fri, 19 Jun 2026 18:59:33 GMT
• Title: Clinical Term Extraction using Open-Source Small Language Models
• Abstract summary: Open-source small language models were evaluated for ALS-relevant clinical terms in patient documentation.<n>We compared 26 open-source models using aggregate, label-level, and manual-validation multilabel classification metrics.<n>These findings support targeted hybrid extraction rather than replacement of existing rule-based methods.
• Score: 2.5316486726943843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clinical information for amyotrophic lateral sclerosis (ALS) care documented in unstructured clinical notes limits downstream analysis without extraction into structured formats. Open-source small language models with few-shot prompting for detecting the presence of ALS-relevant clinical terms in patient documentation were evaluated without task-specific training data. The detection task targeted 17 categories spanning functional scores, respiratory measures, medications, and related clinical and non-clinical attributes. Clinical note content was normalized from JSON-encoded discharge summaries and processed with a prompt template having structured JSON outputs. We compared 26 open-source models using aggregate, label-level, and manual-validation multilabel classification metrics. Manual validation showed that a regex rule baseline had higher overall micro-F1 and lower Hamming loss than any single SLM or TF-IDF baseline, while Qwen3-4B-Instruct-2507 was the highest-performing SLM by micro-F1. Model rankings varied by metric and label category, with the TF-IDF baseline showing high recall but low precision, some SLMs showing higher precision but lower recall, and Hammer2.1-7b showing strong performance for ALSFRS-R subscore detection. These findings support targeted hybrid extraction workflows rather than replacement of existing rule-based methods.

Related papers

• Primary ICD Category Prediction using LLM-based Probing [7.948368617192842]
  We constructed a MIMIC-IV cohort of 13,645 admissions from the 10 most frequent primary ICD-10 codes.<n>We extracted hidden states from five transformer layers and trained linear probes for structured-only, unstructured-only, and combined inputs.<n>The combined probe performed best on MIMIC-IV (87.69% strict; 91.45% medical accuracy), exceeding both single-modality probes and baselines.
  arXiv  Detail & Related papers  (2026-06-27T08:10:05Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Systematic Evaluation of Large Language Models for Post-Discharge Clinical Action Extraction [0.0]
  This paper evaluates zero-shot and few-shot large language models (LLMs) for safety-critical clinical action extraction using the CLIP discharge-note dataset.<n>We introduce a two-stage extraction framework that decomposes discharge notes, that are written in narrative form, into fine-grained, explicitly actionable clinical tasks.
  arXiv  Detail & Related papers  (2026-05-07T13:05:07Z)
• Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction [5.6073733544594395]
  We developed a framework that enables small language models to self-generate, verify, refine, and evaluate entity-specific prompts.<n>We evaluated candidate open-weight models with multi-prompt ensemble inference and selected models using QLoRA-based supervised fine-tuning and direct preference optimization.<n>These findings suggest that automated prompt optimization combined with lightweight preference-based post-training can support scalable clinical information extraction.
  arXiv  Detail & Related papers  (2026-05-05T19:03:40Z)
• CLiGNet: Clinical Label-Interaction Graph Network for Medical Specialty Classification from Clinical Transcriptions [0.0]
  We first document this methodological flaw and establish a leakage free benchmark across 40 medical specialties (4966 records)<n>We then introduce CLiGNet, a neural architecture that combines a Bio ClinicalBERT text encoder with a two layer Graph Convolutional Network operating on a specialty label graph constructed from semantic similarity and ICD 10 chapter priors.<n>We provide comprehensive failure analysis covering pairwise specialty confusions, rare class behaviour, document length effects, and token level Integrated Gradients attribution, offering actionable insights for clinical NLP system deployment.
  arXiv  Detail & Related papers  (2026-03-24T03:30:06Z)
• AgentScore: Autoformulation of Deployable Clinical Scoring Systems [45.88028371034407]
  We introduce AgentScore, which performs semantically guided optimization in unit-weighted clinical checklists.<n>AgentScore outperforms existing score-generation methods and achieves AUC comparable to more flexible interpretable models.<n>On two additional externally validated tasks, AgentScore achieves higher discrimination than established guideline-based scores.
  arXiv  Detail & Related papers  (2026-01-29T21:11:06Z)
• CNSight: Evaluation of Clinical Note Segmentation Tools [3.673249612734457]
  We evaluate rule-based baselines, domain-specific transformer models, and large language models for clinical note segmentation using a curated dataset of 1,000 notes from MIMIC-IV.<n>Our experiments show that large API-based models achieve the best overall performance, with GPT-5-mini reaching a best average F1 of 72.4 across sentence-level and freetext segmentation.
  arXiv  Detail & Related papers  (2025-12-28T05:40:15Z)
• Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction [44.23284500920266]
  This benchmark standardises data curation and evaluation across two distinct clinical settings.<n>Experiments reveal that event stream models consistently deliver the strongest performance.<n>We find that feature selection strategies must be adapted to the clinical setting.
  arXiv  Detail & Related papers  (2025-10-10T09:03:47Z)
• Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation [3.952186976672079]
  We show that lightweight interventions, including prompt engineering and small-scale fine-tuning, can improve accuracy without the computational overhead of search-based methods.<n>To address hierarchically near-miss errors, we introduce clinical code verification as both a standalone task and a pipeline component.
  arXiv  Detail & Related papers  (2025-10-08T23:50:58Z)
• Clinical NLP with Attention-Based Deep Learning for Multi-Disease Prediction [44.0876796031468]
  This paper addresses the challenges posed by the unstructured nature and high-dimensional semantic complexity of electronic health record texts.<n>A deep learning method based on attention mechanisms is proposed to achieve unified modeling for information extraction and multi-label disease prediction.
  arXiv  Detail & Related papers  (2025-07-02T07:45:22Z)
• Multimodal Pretraining of Medical Time Series and Notes [45.89025874396911]
  Deep learning models show promise in extracting meaningful patterns, but they require extensive labeled data. We propose a novel approach employing self-supervised pretraining, focusing on the alignment of clinical measurements and notes. In downstream tasks, including in-hospital mortality prediction and phenotyping, our model outperforms baselines in settings where only a fraction of the data is labeled.
  arXiv  Detail & Related papers  (2023-12-11T21:53:40Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• An Interpretable End-to-end Fine-tuning Approach for Long Clinical Text [72.62848911347466]
  Unstructured clinical text in EHRs contains crucial information for applications including decision support, trial matching, and retrospective research. Recent work has applied BERT-based models to clinical information extraction and text classification, given these models' state-of-the-art performance in other NLP domains. In this work, we propose a novel fine-tuning approach called SnipBERT. Instead of using entire notes, SnipBERT identifies crucial snippets and feeds them into a truncated BERT-based model in a hierarchical manner.
  arXiv  Detail & Related papers  (2020-11-12T17:14:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.