RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding

• URL: http://arxiv.org/abs/2608.00147v1
• Date: Fri, 31 Jul 2026 16:14:24 GMT
• Title: RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
• Abstract summary: We introduce RadPRISM, which makes a clinician-defined radiology schema a designated stratification axis.<n>An on-premise large language model extracts per-concept text spans from free-text reports, and each clinical concept is aligned in its own dedicated visual subspace.
• Score: 14.036871627954156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language pretraining learns rich medical image representations from radiology reports, but previous model variants commonly operate within a single shared embedding space, so concept-level structure and interpretability must be recovered post hoc, limiting model transparency and, hence, clinical utility. We introduce RadPRISM, which makes a clinician-defined radiology schema a designated stratification axis: an on-premise large language model extracts per-concept text spans from free-text reports, and each clinical concept is aligned in its own dedicated visual subspace, turning concept stratification into direct, top-level alignment supervision. Instantiated on chest radiographs with a 19-concept schema over $203{,}602$ examinations from an internal multi-year archive, RadPRISM improved internal dataset zero-shot classification from $0.717$ (95% CI, $0.710-0.723$) to $0.868$ (95% CI, $0.863-0.872$) macro AUROC over a matched global-alignment baseline, performed on par with the purpose-built CARZero reference in external zero-shot classification while substantially outperforming it (up to 4.3-fold) in pointing-game visual grounding. In addition, a radiologist reader study demonstrated concept-stratified retrieval ability ($0.78$ macro retrieval correctness rate within rank 3), surfacing disentangled descriptive findings that report-level retrieval and fixed-label vocabularies cannot express. RadPRISM yields discriminative, spatially faithful, natively concept-stratified representations shaped by and transparently inspectable by clinicians.

Related papers

• CARDEA: Auditable Reasoning Grounded in Spatial Evidence for End-to-End Coronary Angiography Interpretation [0.0]
  Existing AI systems can improve consistency but lack auditable decision processes.<n>We developed CARDEA, a unified large vision-language model that serves as the inference core of a CAG pipeline.<n> CARDEA runs an end-to-end CAG pipeline from raw multiview videos through selection to study-level diagnosis.
  arXiv  Detail & Related papers  (2026-09-07T02:01:27Z)
• A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports [58.14965296904734]
  EndoCLIP is a vision-language foundation model trained on 125,756 lesion-level image-text pairs.<n>It outperforms general-purpose and biomedical vision-language encoders in both zero-shot and linear-probe settings.
  arXiv  Detail & Related papers  (2026-07-30T16:24:36Z)
• DobicVLM: Aligning Chest X-Ray Report Generation with Clinically-Grounded Programmatic Rewards via Group Relative Policy Optimization [0.4418222884390162]
  We introduce DobicVLM, a vision-language model combining supervised fine-tuning on MedGemma-4B with Group Relative Policy Optimization.<n>Our approach uses interpretable, rule-based reward components; structural verification, anatomical checklist, semantic similarity, and length constraints.<n>DobicVLM outperforms Gemini 2.5 Flash across the majority of criteria, achieving the highest impression accuracy (27.2%) and medical terminology (86.5%) compared to both Gemini 2.5 Flash and MedGemma 4B baselines.
  arXiv  Detail & Related papers  (2026-07-21T11:20:09Z)
• CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation [5.995766869240907]
  CogRad is a framework that structures generation around four stages of a radiologist's reading process.<n>On CheXpert Plus, CogRad attains a BLEU-4 of 0.316 and a CIDEr of 0.322, the best scores among the methods we compare against.
  arXiv  Detail & Related papers  (2026-07-04T12:43:04Z)
• A Vision-language Framework for Comparative Reasoning in Radiology [68.52471827773482]
  We formulate radiological comparison as an entity-aware cross-image reasoning problem.<n>We construct a large-scale comparative imaging resource derived from routine image-report pairs.<n>We develop MedReCo, an entity-aware visual encoder for controllable retrieval of clinically analogous cases.
  arXiv  Detail & Related papers  (2026-06-04T17:12:47Z)
• Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance [16.566888073449714]
  Vision--language models (VLMs) for radiology report generation (RRG) can produce long-form chest CT reports from volumetric scans.<n>Existing methods face two key limitations: (i) training supervision is often coarse, aligning a whole CT volume with a full free-text report without explicit alignment for fine-grained attributes or pathology locations.<n>We propose emphDiscrimi Cue-Prompting with Prompt Dropout (DCP-PD), a plug-and-play framework that distills fine-grained cues from free-text reports and uses them to guide report generation.
  arXiv  Detail & Related papers  (2026-04-12T03:25:41Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment [10.67889367763112]
  RadAlign is a novel framework that combines the predictive accuracy of vision-language models with the reasoning capabilities of large language models.<n>Our framework maintains strong clinical interpretability while reducing hallucinations, advancing automated medical imaging and report analysis through integrated predictive and generative AI.
  arXiv  Detail & Related papers  (2025-01-13T17:55:32Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.