Learning Anatomy-Grounded CT Vision-Language Representations with Organ-Hierarchical Report Knowledge

• URL: http://arxiv.org/abs/2607.10953v1
• Date: Sun, 12 Jul 2026 23:09:23 GMT
• Title: Learning Anatomy-Grounded CT Vision-Language Representations with Organ-Hierarchical Report Knowledge
• Abstract summary: We propose OKA-CT, an organ-hierarchical knowledge-augmented framework for CT-report.<n> OKA-CT first converts free-text reports into organ-conditioned knowledge using radiology report parsing.<n>Stage1 injects anatomy-grounded evidence into the CT visual representation through fine-grained organ-conditioned supervision.<n>Stage2 uses organ-specific report evidence to guide structured report-CT contrastive learning.
• Score: 7.277508278429653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical vision-language pretraining (VLP) from paired CT images and radiology reports enables scalable representation learning, but most existing methods align either whole scans with entire reports or local image regions with text fragments. These formulations underuse a key property of radiology reports: findings are organized around anatomical structures, with abnormalities described by organs, disease concepts, locations, and severity-related attributes. We propose OKA-CT, an organ-hierarchical knowledge-augmented framework for CT-report VLP. OKA-CT first converts free-text reports into organ-conditioned knowledge using radiology report parsing and LLM-assisted semantic structuring. The extracted hierarchy is used across two learning stages. Stage~1 injects anatomy-grounded evidence into the CT visual representation through fine-grained organ-conditioned supervision, while Stage~2 uses organ-specific report evidence to guide structured report-CT contrastive learning, where hierarchy-derived semantic soft targets treat non-paired cases with shared organ-level findings as weak semantic positives rather than uniform negatives. A lightweight query-based global branch further aggregates disease-relevant volumetric evidence for whole-scan representation. On CT-RATE and RAD-ChestCT datasets, OKA-CT achieves zero-shot abnormality diagnosis AUROCs of 84.9 and 72.2, outperforming prior CT VLP baselines. Retrieval and patch-occlusion analyses further show improved report-image alignment and stronger sensitivity to disease-associated anatomical regions.

Related papers

• Semantically Calibrated Evidence Composition for CT Vision-Language Learning [4.757530530297623]
  Learning representations from CT-report pairs requires combining whole-volume context with anatomy-specific evidence.<n>We propose SCOPE (Semantic Evidence Of comPosed), a framework for semantically calibrated evidence composition in CT vision-language learning.
  arXiv  Detail & Related papers  (2026-07-31T19:34:31Z)
• Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding [2.9220076568786335]
  OCP-CT is an organ-conditioned pattern-token alignment framework for CT vision-language pretraining.<n>It preserves a stable global CT-report contrastive branch and introduces an organ pattern interface.<n>On the publicly available CT-RATE and RAD-ChestCT benchmarks, OCP-CT achieves average AUROCs of 84.5% and 69.9% for zero-shot abnormality diagnosis.
  arXiv  Detail & Related papers  (2026-07-15T14:36:43Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT [29.0378459959757]
  EXACT is an anomaly-aware foundation model for three-dimensional chest CT.<n>It learns spatially resolved representations from paired clinical scans and radiology reports.<n> EXACT shows consistent improvements across clinically relevant CT tasks.
  arXiv  Detail & Related papers  (2026-04-27T07:57:47Z)
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• REACT-KD: Region-Aware Cross-modal Topological Knowledge Distillation for Interpretable Medical Image Classification [2.384306241677947]
  We present REACT-KD, a framework that transfers supervision from high-fidelity multi-modal sources into a lightweight CT-based student model.<n>One branch captures structure-function relationships through dual-tracer PET/CT, while the other models dose-aware features using synthetically degraded low-dose CT.<n>The framework achieved an average AUC of 93.5% on an internal PET/CT cohort and maintained 76.6% to 81.5% AUC across varying levels of dose degradation in external CT testing.
  arXiv  Detail & Related papers  (2025-08-04T06:29:34Z)
• A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation [4.408787333571913]
  We propose a framework for automatic renal CT report generation.<n>In Stage 1, a multi-task learning model detects structured clinical features from each 2D image.<n>In Stage 2, a vision-language model generates free-text reports conditioned on the image and the detected features.
  arXiv  Detail & Related papers  (2025-06-30T07:45:02Z)
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis [56.57177181778517]
  RadGenome-Chest CT is a large-scale, region-guided 3D chest CT interpretation dataset based on CT-RATE. We leverage the latest powerful universal segmentation and large language models to extend the original datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:11:37Z)
• Structure-aware registration network for liver DCE-CT images [50.28546654316009]
  We propose a novel structure-aware registration method by incorporating structural information of related organs with segmentation-guided deep registration network. Our proposed method can achieve higher registration accuracy and preserve anatomical structure more effectively than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-08T14:08:56Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)
• Deep Reinforcement Learning for Organ Localization in CT [59.23083161858951]
  We propose a deep reinforcement learning approach for organ localization in CT. In this work, an artificial agent is actively self-taught to localize organs in CT by learning from its asserts and mistakes. Our method can use as a plug-and-play module for localizing any organ of interest.
  arXiv  Detail & Related papers  (2020-05-11T10:06:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.