Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding

• URL: http://arxiv.org/abs/2607.13892v1
• Date: Wed, 15 Jul 2026 14:36:43 GMT
• Title: Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
• Abstract summary: OCP-CT is an organ-conditioned pattern-token alignment framework for CT vision-language pretraining.<n>It preserves a stable global CT-report contrastive branch and introduces an organ pattern interface.<n>On the publicly available CT-RATE and RAD-ChestCT benchmarks, OCP-CT achieves average AUROCs of 84.5% and 69.9% for zero-shot abnormality diagnosis.
• Score: 2.9220076568786335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Computed tomography (CT) vision-language pretraining from paired volumes and radiology reports is a scalable yet challenging task. Existing methods commonly adopt global scan-report contrast, which is scalable but obscures heterogeneous organ evidence. Meanwhile, direct organ-level alignment remains coarse, since the same anatomy can exhibit multiple distinct radiological appearances. Therefore, pretraining requires a finer alignment unit: the organ-conditioned radiological pattern. In this work, we propose OCP-CT, an organ-conditioned pattern-token alignment framework for CT vision-language pretraining. Specifically, OCP-CT preserves a stable global CT-report contrastive branch and introduces an organ pattern interface: sparse Mixture-of-Experts (MoE) routes image and text tokens according to latent radiological patterns, learnable slots query the routed tokens into continuous pattern tokens, and paired token contrast aligns image-text pattern tokens with structured soft targets built from report-derived clinical similarity. On the publicly available CT-RATE and RAD-ChestCT benchmarks, OCP-CT achieves average AUROCs of 84.5% and 69.9% for zero-shot abnormality diagnosis, respectively. Compared with the strongest prior reported results, these results yield absolute AUROC gains of 6.7 and 0.8 percentage points.

Related papers

• LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3 [3.882636699866806]
  We propose a cycle consistent style translation framework built on a shared frozen DINOv3 with modality-specific LoRA adapters.<n>We evaluate translation quality using Fréchet Inception Distance (FID) and structural fidelity via mutual information.<n>As a preliminary indicator of downstream registration utility, we find that style-translated images yield increased feature correspondences under MatchAnything on manually aligned HiP-CT and histology test pairs.
  arXiv  Detail & Related papers  (2026-08-07T15:43:55Z)
• Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining [4.757530530297623]
  Volumetric CT vision-language pretraining learns 3D representations from scan-report pairs.<n>Nothing separates a mild from an extensive case of the same finding along a consistent direction.<n>Cross-sectional cohorts already encode weak burden cues across different patients.
  arXiv  Detail & Related papers  (2026-07-31T19:23:02Z)
• Learning Anatomy-Grounded CT Vision-Language Representations with Organ-Hierarchical Report Knowledge [7.277508278429653]
  We propose OKA-CT, an organ-hierarchical knowledge-augmented framework for CT-report.<n> OKA-CT first converts free-text reports into organ-conditioned knowledge using radiology report parsing.<n>Stage1 injects anatomy-grounded evidence into the CT visual representation through fine-grained organ-conditioned supervision.<n>Stage2 uses organ-specific report evidence to guide structured report-CT contrastive learning.
  arXiv  Detail & Related papers  (2026-07-12T23:09:23Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• Clinically-aligned ischemic stroke segmentation and ASPECTS scoring on NCCT imaging using a slice-gated loss on foundation representations [3.186130813218338]
  Most deep learning methods perform pixel-wise segmentation without modeling the structured anatomical reasoning underlying ASPECTS scoring.<n>We propose a clinically aligned framework that combines a frozen DINOv3 backbone with a lightweight decoder.<n>Our method achieves a Dice score of 0.6385 on AISD, outperforming prior CNN and foundation-model baselines.
  arXiv  Detail & Related papers  (2026-02-27T12:12:24Z)
• Hide-and-Seek Attribution: Weakly Supervised Segmentation of Vertebral Metastases in CT [68.09387763135236]
  We introduce a weakly supervised method trained solely on vertebra-level healthy/malignant labels, without any lesion masks.<n>We achieve strong blastic/lytic performance despite no mask supervision.
  arXiv  Detail & Related papers  (2025-12-07T14:03:28Z)
• SimCroP: Radiograph Representation Learning with Similarity-driven Cross-granularity Pre-training [25.763109982379703]
  We propose a Similarity-Driven Cross-Granularity Pre-training framework on chest CTs.<n>It combines similarity-driven alignment and cross-granularity fusion to improve radiograph interpretation.<n>SimCroP is pre-trained on a large-scale paired CT-reports dataset and validated on image classification and segmentation tasks.
  arXiv  Detail & Related papers  (2025-09-10T06:20:53Z)
• Class-Aware Cartilage Segmentation for Autonomous US-CT Registration in Robotic Intercostal Ultrasound Imaging [39.597735935731386]
  A class-aware cartilage bone segmentation network with geometry-constraint post-processing is presented to capture patient-specific rib skeletons. A dense skeleton graph-based non-rigid registration is presented to map the intercostal scanning path from a generic template to individual patients. Results demonstrate that the proposed graph-based registration method can robustly and precisely map the path from CT template to individual patients.
  arXiv  Detail & Related papers  (2024-06-06T14:15:15Z)
• CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
  We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning. Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
  arXiv  Detail & Related papers  (2024-04-23T17:59:01Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation [4.3971310109651665]
  In current clinical practice, noisy and artifact-ridden weekly cone-beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is done once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment and for deriving biomarkers for treatment response.
  arXiv  Detail & Related papers  (2021-03-09T19:51:44Z)
• Detecting Pancreatic Ductal Adenocarcinoma in Multi-phase CT Scans via Alignment Ensemble [77.5625174267105]
  Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers among the population. Multiple phases provide more information than single phase, but they are unaligned and inhomogeneous in texture. We suggest an ensemble of all these alignments as a promising way to boost the performance of PDAC detection.
  arXiv  Detail & Related papers  (2020-03-18T19:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.