Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models

• URL: http://arxiv.org/abs/2606.18749v1
• Date: Wed, 17 Jun 2026 06:52:07 GMT
• Title: Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models
• Abstract summary: We propose CS3F, a training-free framework for anomaly detection in 3D medical images using 2D foundation models.<n>Anomaly scores are obtained from cross-subject mutual similarity.<n>CS3F is evaluated on brain MRI across metastases, glioma, and stroke, as well as validated on lung CT to test generalizability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-shot anomaly detection (ZSAD) is attractive for medical imaging because clinical systems must handle heterogeneous acquisition protocols, changing patient populations, and pathologies for which annotated training data may be unavailable. Most existing zero-shot anomaly detection methods are designed for 2D images, and their direct extension to 3D medical volumes is limited by the scarcity of large-scale volumetric foundation models or by the difficulty of utilizing volumetric context. We propose CS3F, a training-free batch-based framework for ZSAD in 3D medical images using 2D foundation models. Each volume is decomposed along multiple anatomical axes and encoded slice-wise by a 2D vision transformer. These are then converted into localized volumetric tokens by pooling neighboring slice features. Anomaly scores are obtained from cross-subject mutual similarity: tokens that lack close analogues in other subjects are assigned higher anomaly scores. To reduce the attenuation of focal lesion signals caused by depth pooling, we introduce a coarse-to-fine tokenization strategy that enables fine-resolution volumetric scoring without exhaustive matching. CS3F is evaluated on brain MRI across metastases, glioma, and stroke, as well as validated on lung CT to test generalizability beyond atlas-aligned brain MRI. The results show that frozen 2D foundation models can support anomaly localization in 3D medical images, and that the benefit of fine tokenization depends strongly on lesion contrast and imaging modality.

Related papers

• Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models [0.0]
  We introduce a fully training-free framework for ZSAD in 3D brain MRI.<n>The framework constructs localized volumetric tokens by aggregating multi-axis slices processed by 2D foundation models.<n>These 3D patch tokens restore cubic spatial context and integrate directly with distance-based, batch-level anomaly detection pipelines.
  arXiv  Detail & Related papers  (2026-02-17T02:46:45Z)
• Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity [47.52394044948656]
  Inter-Slice Consistentity (ISCS) is a strategy that encourages interslice consistency during diffusion sampling.<n>ISCS can be dropped into any 2D trained diffusion based 3D reconstruction pipeline without additional computational cost.
  arXiv  Detail & Related papers  (2026-02-04T02:54:08Z)
• Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification [69.87877580725768]
  Multimodal Visual Surrogate Compression (MVSC) learns to compress and adapt large 3D sMRI volumes into compact 2D features.<n>MVSC has two key components: a Volume Context that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner.
  arXiv  Detail & Related papers  (2026-01-29T13:05:46Z)
• Don't Mind the Gaps: Implicit Neural Representations for Resolution-Agnostic Retinal OCT Analysis [1.0093567618205226]
  Implicit neural representations (INRs) have emerged as a tool to store voxelized data as a continuous representation.<n>We propose two frameworks that make use of this characteristic of INRs for dense 3D analyses of retinal OCT volumes.
  arXiv  Detail & Related papers  (2026-01-05T15:41:03Z)
• Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning [0.15749416770494706]
  Self-supervised deep learning has accelerated 2D natural image analysis but remains difficult to translate into 3D MRI.<n>We present a emph-sequence-invariant self-supervised framework leveraging quantitative MRI (qMRI)<n> Experiments on healthy brain segmentation (IXI), stroke lesion segmentation (ARC), and MRI denoising show significant gains over baseline SSL approaches.
  arXiv  Detail & Related papers  (2025-01-21T11:27:54Z)
• CABLD: Contrast-Agnostic Brain Landmark Detection with Consistency-Based Regularization [2.423045468361048]
  We introduce CABLD, a novel self-supervised deep learning framework for 3D brain landmark detection in unlabeled scans.<n>We demonstrate the proposed method with the intricate task of MRI-based 3D brain landmark detection.<n>Our framework provides a robust and accurate solution for anatomical landmark detection, reducing the need for extensively annotated datasets.
  arXiv  Detail & Related papers  (2024-11-26T19:56:29Z)
• Abnormality-Driven Representation Learning for Radiology Imaging [0.8321462983924758]
  We introduce lesion-enhanced contrastive learning (LeCL), a novel approach to obtain visual representations driven by abnormalities in 2D axial slices across different locations of the CT scans. We evaluate our approach across three clinical tasks: tumor lesion location, lung disease detection, and patient staging, benchmarking against four state-of-the-art foundation models.
  arXiv  Detail & Related papers  (2024-11-25T13:53:26Z)
• X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Models [9.97747208739885]
  X-Diffusion is a novel cross-sectional diffusion model that reconstructs detailed 3D MRI volumes from extremely sparse spatial-domain inputs.<n>A key aspect of X-Diffusion is that it models MRI data as holistic 3D volumes during the cross-sectional training and inference.<n>Our results demonstrate that X-Diffusion not only surpasses state-of-the-art methods in quantitative accuracy (PSNR) on unseen data but also preserves critical anatomical features.
  arXiv  Detail & Related papers  (2024-04-30T14:53:07Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images [39.94162291765236]
  We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process.
  arXiv  Detail & Related papers  (2023-08-03T21:56:50Z)
• Explainable multiple abnormality classification of chest CT volumes with AxialNet and HiResCAM [89.2175350956813]
  We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images. We propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. We then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions.
  arXiv  Detail & Related papers  (2021-11-24T01:14:33Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE [66.63629641650572]
  We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices. We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
  arXiv  Detail & Related papers  (2020-07-09T13:23:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.