An end-to-end-trained vision-language model for native-language prostate pathology report generation

• URL: http://arxiv.org/abs/2608.23143v1
• Date: Mon, 24 Aug 2026 11:49:23 GMT
• Title: An end-to-end-trained vision-language model for native-language prostate pathology report generation
• Abstract summary: Prostate cancer is among the most frequently diagnosed malignancies worldwide.<n>Existing tools frame this as classification, leaving pathologists to assemble coherent reports.<n>We present a slide-level framework generating prostate biopsy reports that is language-independent by construction.
• Score: 0.09772267314090433
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Prostate cancer is among the most frequently diagnosed malignancies worldwide, and structured reporting of each biopsy core burdens pathologists. Existing tools frame this as classification, leaving pathologists to assemble coherent reports, while many slide-level vision-language models rely on English-centric encoders that transfer poorly to other clinical languages. We present a slide-level framework generating prostate biopsy reports that is language-independent by construction: tokenizer and model are trained from scratch, demonstrated here in German. To address paired-data scarcity, an automated pipeline uses a locally deployed large language model to split composite reports into core-specific image-text pairs, yielding 17,344 pairs from 2,402 historical cases without manual annotation. Evaluated for clinical attributes rather than linguistic similarity, the model achieves 96.2% F1 for malignancy detection and 65.2% for Gleason grading, competitive with an FDA-cleared classifier. Grading is further validated on three external cohorts with latent-space augmentation. Institutions can thus train native-language reporting models on their own archives.

Related papers

• A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports [58.14965296904734]
  EndoCLIP is a vision-language foundation model trained on 125,756 lesion-level image-text pairs.<n>It outperforms general-purpose and biomedical vision-language encoders in both zero-shot and linear-probe settings.
  arXiv  Detail & Related papers  (2026-07-30T16:24:36Z)
• PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology [46.36925477064605]
  PanDent is a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations.<n>The dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations.<n>Results show that current MLLMs can generate fluent reports, yet fail to produce clinically consistent descriptions.
  arXiv  Detail & Related papers  (2026-07-29T18:36:07Z)
• Generating Reports or Repeating Templates? Measuring and Mitigating Template Collapse in 3D CT Report Generation [13.471480514512157]
  3D medical vision-language models (VLMs) can generate fluent radiology-style text while exhibit critically low pathology detection and output diversity.<n>We identify this failure mode as template collapse.<n>We propose CLarGen, a framework that separates what to say (clinical detection) from how to say it (language synthesis)
  arXiv  Detail & Related papers  (2026-05-29T08:21:00Z)
• Can Large Language Models Reliably Extract Physiology Index Values from Coronary Angiography Reports? [1.1061680331694044]
  This study is the first addressing physiology indexes extraction from a large (1342 reports) corpus of CAG reports.<n>Given the sparsity of measurements, we propose a multi-stage evaluation framework separating format validity, value detection, and value correctness.
  arXiv  Detail & Related papers  (2026-03-21T01:33:18Z)
• A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation [4.408787333571913]
  We propose a framework for automatic renal CT report generation.<n>In Stage 1, a multi-task learning model detects structured clinical features from each 2D image.<n>In Stage 2, a vision-language model generates free-text reports conditioned on the image and the detected features.
  arXiv  Detail & Related papers  (2025-06-30T07:45:02Z)
• On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation [0.7966328552094392]
  Vision-language models in pathology enable multimodal case retrieval and automated report generation.<n>Many of the models developed so far have been trained on pathology reports that include information which cannot be inferred from paired whole slide images.<n>We investigate how the selection of information from pathology reports for vision-language modeling affects the quality of the multimodal representations and generated reports.
  arXiv  Detail & Related papers  (2025-02-26T16:45:09Z)
• Self-supervised vision-langage alignment of deep learning representations for bone X-rays analysis [53.809054774037214]
  This paper proposes leveraging vision-language pretraining on bone X-rays paired with French reports. It is the first study to integrate French reports to shape the embedding space devoted to bone X-Rays representations.
  arXiv  Detail & Related papers  (2024-05-14T19:53:20Z)
• Hierarchical Classification System for Breast Cancer Specimen Report (HCSBC) -- an end-to-end model for characterizing severity and diagnosis [3.4454444815042735]
  We develop a hierarchical hybrid transformer-based pipeline (59 labels) - Hierarchical Classification System for Breast Cancer Specimen Report (HCSBC) We trained the model on the EUH data and evaluated our model's performance on two external datasets - MGH and Mayo Clinic.
  arXiv  Detail & Related papers  (2023-11-02T18:37:45Z)
• Evaluating Large Language Models for Radiology Natural Language Processing [68.98847776913381]
  The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP) This study seeks to bridge this gap by critically evaluating thirty two LLMs in interpreting radiology reports.
  arXiv  Detail & Related papers  (2023-07-25T17:57:18Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.