Sparse Concept Channels in Frozen 3D CT Vision Encoders

• URL: http://arxiv.org/abs/2607.20993v1
• Date: Thu, 23 Jul 2026 07:17:11 GMT
• Title: Sparse Concept Channels in Frozen 3D CT Vision Encoders
• Abstract summary: Large vision-language models are becoming increasingly dominant in 3D medical image interpretation.<n>We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings.<n>We show that (i) each radiological finding is encoded by a i>sparse/i> set of 10 vision-encoder channels that match full-feature classification performance.
• Score: 3.9173955300197973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large vision-language models are becoming increasingly dominant in 3D medical image interpretation, but we rarely know <i>which</i> internal units encode clinical findings or <i>where</i> that information lives in the representation. We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings. We show that (i) each radiological finding is encoded by a <i>sparse</i> set of ~10 vision-encoder channels that match full-feature classification performance and far exceed a zero-shot text prompting; (ii) turning off the channels tied to one finding, that finding's score collapses while unrelated labels stay stable; and (iii) the same sparse probe <i>replicates</i> on an architecturally unrelated 3D abdominal VLM (Merlin) suggesting a general property of frozen medical encoders. Our training-free concept channel probe (CCP) method, paired with a corpus-derived report template, outperforms published CT-CHAT on clinical efficacy and NLG metrics (F1 0.549 vs. 0.184; BLEU 0.483 vs. 0.373) at 22x lower latency. Our results provide a clear, reproducible characterization of how frozen medical encoders represent findings, demonstrating direct applicability across models.

Related papers

• Frozen Brain-MRI Foundation Models Are Site Fingerprints [0.0]
  Frozen foundation-model embeddings are increasingly used as off-the-shelf brain-MRI representations.<n>We audit what they actually encode and find that site is a large, intrinsic component of the representation.<n>We recommend site-audited use of frozen brain-MRI FMs and release an open audit toolkit.
  arXiv  Detail & Related papers  (2026-08-10T23:03:21Z)
• Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models [30.102620002840904]
  3D CT foundation models could assist routine interpretation by providing generalizable representations of anatomy and pathology.<n>We benchmark ten frozen CT encoders across three cohorts of thoracic CT scans.<n>We find no universal state-of-the-art, with rankings fluctuating significantly depending on the evaluation context.
  arXiv  Detail & Related papers  (2026-08-06T12:34:17Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• DINO-MVR: Multi-View Readout of Frozen DINOv3 for Annotation-Efficient Medical Segmentation [10.929173865297933]
  We show that frozen DINOv3 features already contain useful structural and boundary cues for medical segmentation.<n>We propose DINO-MVR, a Multi-View Readout framework for annotation-efficient medical segmentation.
  arXiv  Detail & Related papers  (2026-05-08T04:13:01Z)
• U-VLM: Hierarchical Vision Language Modeling for Report Generation [20.09433657986766]
  We propose U-VLM, which enables hierarchical vision-language modeling in both training and architecture.<n>U-VLM achieves state-of-the-art performance on CT-RATE and AbdomenAtlas 3.0 using only a 0.1B decoder trained from scratch.
  arXiv  Detail & Related papers  (2026-02-28T05:43:11Z)
• An Explainable Non-local Network for COVID-19 Diagnosis [37.378584156643825]
  We propose a novel deep residual 3D attention non-local network (NL-RAN) to classify CT images included COVID-19, common pneumonia, and normal. The network is embedded with a nonlocal module to capture global information, while a 3D attention module is embedded to focus on the details of the lesion. Our experimental results indicate that our proposed method performs significantly better than existing methods.
  arXiv  Detail & Related papers  (2024-08-08T08:35:21Z)
• Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images [45.29301790646322]
  Computer-aided diagnosis can help with early lung nodul detection and facilitate subsequent nodule characterization. We propose CADe, for segmenting lung nodules in a zero-shot manner using a variant of the Segment Anything Model called MedSAM. We also propose, CADx, a method for the nodule characterization as benign/malignant by making a gallery of radiomic features and aligning image-feature pairs through contrastive learning.
  arXiv  Detail & Related papers  (2024-07-02T19:30:25Z)
• CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
  We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning. Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
  arXiv  Detail & Related papers  (2024-04-23T17:59:01Z)
• WATUNet: A Deep Neural Network for Segmentation of Volumetric Sweep Imaging Ultrasound [1.2903292694072621]
  Volume sweep imaging (VSI) is an innovative approach that enables untrained operators to capture quality ultrasound images. We present a novel segmentation model known as Wavelet_Attention_UNet (WATUNet) In this model, we incorporate wavelet gates (WGs) and attention gates (AGs) between the encoder and decoder instead of a simple connection to overcome the limitations mentioned.
  arXiv  Detail & Related papers  (2023-11-17T20:32:37Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• COVID-19 identification from volumetric chest CT scans using a progressively resized 3D-CNN incorporating segmentation, augmentation, and class-rebalancing [4.446085353384894]
  COVID-19 is a global pandemic disease overgrowing worldwide. Computer-aided screening tools with greater sensitivity is imperative for disease diagnosis and prognosis. This article proposes a 3D Convolutional Neural Network (CNN)-based classification approach.
  arXiv  Detail & Related papers  (2021-02-11T18:16:18Z)
• Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans [72.04652116817238]
  We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification. We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
  arXiv  Detail & Related papers  (2021-01-14T03:45:01Z)
• Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction [57.74609918453932]
  Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses. Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images. We argue that preserving the continuity of extracted vessels requires to take into account the global geometry. We propose a hybrid representation learning approach to address this challenge.
  arXiv  Detail & Related papers  (2020-12-14T05:22:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.