Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives

• URL: http://arxiv.org/abs/2606.19852v2
• Date: Thu, 25 Jun 2026 12:44:39 GMT
• Title: Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives
• Abstract summary: Key data remains embedded in narrative reports, making manual extraction labor-intensive and error-prone.<n>We developed a zero-shot, agentic workflow to populate 13 College of American Pathologists synoptic fields from lung resection pathology reports.<n>The best zero-shot model (GPT-OSS-20B) achieved Micro-F1 of 0.893, accurately extracting complex relations like Pathologic Stage without task-specific training.
• Score: 10.547193480104488
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Information extraction from pathology reports is essential for cancer staging, tumor registry population. Yet key data remains embedded in narrative reports, making manual extraction labor-intensive and error-prone. Traditional supervised Natural Language Processing pipelines address this through fully supervised Named Entity Recognition and Relation Extraction, but require expensive manual annotation and suffer cascading failures when upstream entities are missed. In this study, we developed a zero-shot, agentic workflow, and evaluated five open-source generative Large Language Models (LLMs) to populate 13 College of American Pathologists synoptic fields from lung resection pathology reports. We compared them against a state-of-the-art supervised GatorTron NER-RE baseline using a novel, registry-aligned evaluation framework. The baseline achieved Micro-F1of 0.960, while the best zero-shot model (GPT-OSS-20B) achieved Micro-F1 of 0.893 (recall: 0.949), accurately extracting complex relations like Pathologic Stage without task-specific training. These results suggest that open-source, zero-shot agentic LLMs show great potential as a low-cost solution for extracting lung pathology information.

Related papers

• Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extraction [6.53191669778827]
  This study introduces a parameter-efficient, multi-task framework for automating the extraction of Tumor-Node-Metastasis (TNM) staging, histologic grade, and biomarkers.<n>We fine-tune a Llama-3-8B-Instruct encoder using Low-Rank Adaptation (LoRA) on a curated, expert-verified dataset of 10,677 reports.
  arXiv  Detail & Related papers  (2026-04-14T22:26:30Z)
• Tracking Cancer Through Text: Longitudinal Extraction From Radiology Reports Using Open-Source Large Language Models [0.6642706656153227]
  We present a fully open-source, locally deployable pipeline for longitudinal information extraction from radiology reports.<n> Evaluation on 50 Dutch CT Thorax/Abdomen report pairs yielded high extraction performance, with attribute-level accuracies of 93.7% for target lesions, 94.9% for non-target lesions, and 94.0% for new lesions.
  arXiv  Detail & Related papers  (2026-03-10T13:13:43Z)
• PathoSCOPE: Few-Shot Pathology Detection via Self-Supervised Contrastive Learning and Pathology-Informed Synthetic Embeddings [42.42150241818321]
  Unsupervised pathology detection trains models on non-pathological data to flag deviations as pathologies.<n>We propose PathoSCOPE, a few-shot unsupervised pathology detection framework that requires only a small set of non-pathological samples.<n>PathoSCOPE achieves state-of-the-art performance among unsupervised methods while maintaining computational efficiency (2.48 GFLOPs, 166 FPS)
  arXiv  Detail & Related papers  (2025-05-23T08:21:58Z)
• A Benchmark for End-to-End Zero-Shot Biomedical Relation Extraction with LLMs: Experiments with OpenAI Models [8.888403936226185]
  We study patterns in the performance of OpenAI LLMs across a diverse sampling of biomedical relation extraction tasks.<n>Our work is the first to study and compare the performance of the GPT-4, o1 and GPT-OSS for the end-to-end zero-shot biomedical RE task.
  arXiv  Detail & Related papers  (2025-04-05T07:08:54Z)
• Evaluating Large Language Models for Automated Clinical Abstraction in Pulmonary Embolism Registries: Performance Across Model Sizes, Versions, and Parameters [16.74673750576054]
  We evaluated whether openly available large-language models (LLMs) can automate concept extraction from computed-tomography PE (CTPE) reports without sacrificing data quality.<n>LLMs offer a scalable, accurate solution for PE registry abstraction, and a dual-model review workflow can further safeguard data quality with minimal human oversight.
  arXiv  Detail & Related papers  (2025-03-26T21:38:06Z)
• Leveraging large language models for structured information extraction from pathology reports [0.0]
  We evaluate large language models' accuracy in extracting structured information from breast cancer histopathology reports.<n>Open-source tool for structured information extraction can be customized by non-programmers using natural language.
  arXiv  Detail & Related papers  (2025-02-14T21:46:02Z)
• Phikon-v2, A large and public feature extractor for biomarker prediction [42.52549987351643]
  We train a vision transformer using DINOv2 and publicly release one iteration of this model for further experimentation, coined Phikon-v2. While trained on publicly available histology slides, Phikon-v2 surpasses our previously released model (Phikon) and performs on par with other histopathology foundation models (FM) trained on proprietary data.
  arXiv  Detail & Related papers  (2024-09-13T20:12:29Z)
• Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images [45.29301790646322]
  Computer-aided diagnosis can help with early lung nodul detection and facilitate subsequent nodule characterization. We propose CADe, for segmenting lung nodules in a zero-shot manner using a variant of the Segment Anything Model called MedSAM. We also propose, CADx, a method for the nodule characterization as benign/malignant by making a gallery of radiomic features and aligning image-feature pairs through contrastive learning.
  arXiv  Detail & Related papers  (2024-07-02T19:30:25Z)
• Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers [0.29530625605275984]
  structured reporting (SR) has been recommended by various medical societies. We propose a pipeline to extract information from free-text reports. Our work aims to leverage the potential of Natural Language Processing (NLP) and Transformer-based models.
  arXiv  Detail & Related papers  (2024-03-27T18:38:39Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)
• DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature [143.5381108333212]
  We show that text sampled from an large language model tends to occupy negative curvature regions of the model's log probability function. We then define a new curvature-based criterion for judging if a passage is generated from a given LLM. We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection.
  arXiv  Detail & Related papers  (2023-01-26T18:44:06Z)
• WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma [51.50991881342181]
  This challenge includes 10,091 patch-level annotations and over 130 million labeled pixels. First place team achieved mIoU of 0.8413 (tumor: 0.8389, stroma: 0.7931, normal: 0.8919)
  arXiv  Detail & Related papers  (2022-04-13T15:27:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.