Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models

• URL: http://arxiv.org/abs/2608.05960v1
• Date: Thu, 06 Aug 2026 12:34:17 GMT
• Title: Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models
• Abstract summary: 3D CT foundation models could assist routine interpretation by providing generalizable representations of anatomy and pathology.<n>We benchmark ten frozen CT encoders across three cohorts of thoracic CT scans.<n>We find no universal state-of-the-art, with rankings fluctuating significantly depending on the evaluation context.
• Score: 30.102620002840904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Routine CT interpretation is inherently comprehensive, capturing incidental findings across the entire scan volume. 3D CT foundation models could assist this process by providing generalizable representations of anatomy and pathology. To evaluate their diagnostic breadth, we benchmark ten frozen CT encoders across three cohorts of thoracic CT scans, including an unseen internal clinical dataset, using $k$-nearest neighbors, zero-shot prompting, and linear probing. We find no universal state-of-the-art, with rankings fluctuating significantly depending on the evaluation context. While models combining fine-grained image tokenization with vision-language alignment generally perform best, a lightweight supervised encoder remains highly competitive, demonstrating that explicit labels can effectively substitute for scale. Crucially, rather than model architecture, we observe that the primary determinant of performance is a physical bottleneck: a finding's detectability scales with its contrast against surrounding tissue and its spatial extent. Through controlled within-organ comparisons, we empirically demonstrate that widespread or high-contrast abnormalities, such as devices and effusions, are reliably recovered. Conversely, small, low-contrast focal lesions remain a persistent challenge across all evaluated encoders. We attribute this to the inherent limitations of globally pooled embeddings, suggesting that accurately representing small, low-contrast structures will require region- or lesion-level pretraining.

Related papers

• Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography [10.665332135327205]
  We introduce Rad-JEPA 3D, a joint-embedding predictive framework that learns volumetric CT representations.<n>To improve the quality of intermediate representations, we propose Hidden States Orthogonal Regularization (HSOR)<n>HSOR aligns student-teacher hidden states and reduces feature redundancy throughout the encoder.
  arXiv  Detail & Related papers  (2026-07-28T19:00:17Z)
• MAC-XA: Multi-view Anatomy-Correspondence Fusion for Coronary Stenosis Reporting from X-ray Angiography [46.98348758460767]
  Multi-view reasoning in coronary X-ray angiography is inherently a cross-projection geometric problem.<n>We introduce a controllable synthetic angiography generation strategy to expose geometry-derived patch-level correspondence supervision.<n>Experiments on synthetic data and zero-shot transfer to real angiograms show that this alignment-constrained design improves correspondence consistency and structured stenosis reporting.
  arXiv  Detail & Related papers  (2026-07-07T13:33:25Z)
• 3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance [0.0]
  Ultrasound (US) is non-ionizing and widely accessible; however, it is highly operator dependent and lacks quantitative tissue characterization.<n>This work presents a 3D ultrasound-derived pseudo-CT framework that generates CT-like anatomical reference volumes inferred from US.<n>The proposed method outperforms established baselines in structural fidelity and perceptual image quality.
  arXiv  Detail & Related papers  (2026-05-06T12:52:59Z)
• EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT [29.0378459959757]
  EXACT is an anomaly-aware foundation model for three-dimensional chest CT.<n>It learns spatially resolved representations from paired clinical scans and radiology reports.<n> EXACT shows consistent improvements across clinically relevant CT tasks.
  arXiv  Detail & Related papers  (2026-04-27T07:57:47Z)
• PHASOR: Anatomy- and Phase-Consistent Volumetric Diffusion for CT Virtual Contrast Enhancement [60.81732730684265]
  This paper introduces PHASOR, a volumetric diffusion framework for high-fidelity virtual contrast enhancement (VCE)<n>By treating CT volumes as coherent sequences, we leverage a video diffusion model to enhance structural coherence and volumetric accuracy.<n>To ensure anatomy-phase consistent synthesis, we introduce two complementary modules. First, anatomy-routed mixture-of-experts (AR-MoE) anchors distinct enhancement patterns to anatomical semantics, with organ-specific memory to capture salient details.<n>Second, intensity-phase aware representation alignment (IP-REPA) highlights intricate contrast signals while mitigating the impact of imperfect spatial alignment.
  arXiv  Detail & Related papers  (2026-04-01T15:57:18Z)
• Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers [57.54843029965778]
  Controllable pathology image synthesis requires reliable regulation of spatial layout, tissue morphology, and semantic detail.<n>We propose In-Context Diffusion Transformer (IC-DiT), a layout-aware generative model that incorporates spatial layouts, textual descriptions, and visual embeddings into a unified diffusion transformer.<n>IC-DiT achieves higher fidelity, stronger spatial controllability, and better diagnostic consistency than existing methods.
  arXiv  Detail & Related papers  (2026-03-11T06:14:11Z)
• Organ-Aware Attention Improves CT Triage and Classification [0.7901846308308808]
  Off-the-shelf Vision Language Models struggle with 3D anatomy, protocol shifts, and noisy report supervision.<n>This study used the two largest publicly available chest CT datasets: CT-RATE and RADCHEST-CT.<n>We present ORACLE-CT, an encoder-agnostic, organ-aware head that pairs Organ-Masked Attention with Organ-Scalar Fusion.
  arXiv  Detail & Related papers  (2026-01-19T20:37:45Z)
• Unified Supervision For Vision-Language Modeling in 3D Computed Tomography [1.4193731654133002]
  General-purpose vision-language models (VLMs) have emerged as promising tools in radiology, offering zero-shot capabilities.<n>In high-stakes domains like diagnostic radiology, these models often lack the discriminative precision required for reliable clinical use.<n>We introduce Uniferum, a volumetric VLM that unifies diverse supervision signals, encoded in classification labels and segmentation masks, into a single training framework.
  arXiv  Detail & Related papers  (2025-09-01T15:30:17Z)
• Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach [57.86418347491272]
  We propose a comprehensive hierarchical classification system, with 404 representative abnormal findings across all body regions.<n>We contribute a dataset containing over 14.5K CT images from multiple planes and all human body regions, and meticulously provide grounding annotations for over 19K abnormalities.<n>We propose OminiAbnorm-CT, which can automatically ground and describe abnormal findings on multi-plane and whole-body CT images based on text queries.
  arXiv  Detail & Related papers  (2025-06-03T17:57:34Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• Hepatic vessel segmentation based on 3Dswin-transformer with inductive biased multi-head self-attention [46.46365941681487]
  We propose a robust end-to-end vessel segmentation network called Indu BIased Multi-Head Attention Vessel Net. We introduce the voxel-wise embedding rather than patch-wise embedding to locate precise liver vessel voxels. On the other hand, we propose inductive biased multi-head self-attention which learns inductive biased relative positional embedding from absolute position embedding.
  arXiv  Detail & Related papers  (2021-11-05T10:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.