Knowledge Transfer Scaling Laws for 3D Medical Imaging

• URL: http://arxiv.org/abs/2605.06859v1
• Date: Thu, 07 May 2026 19:00:12 GMT
• Title: Knowledge Transfer Scaling Laws for 3D Medical Imaging
• Abstract summary: We show that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric.<n>Motivated by these findings, we formulate data allocation as a scaling-law optimization problem.<n>Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58%.
• Score: 17.692545975214795
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across different imaging modalities (i.e. CT, MRI, and PET) could provide foundational models for diverse clinical tasks. However, training such models requires mixing heterogeneous imaging domains, and current mixture strategies remain largely heuristic. In this work, we observe that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric: training on one domain can substantially improve another, but the reverse may be much weaker. Interestingly, both MAE reconstruction loss and cross-domain transfer follow predictable power-law trends with domain-specific behaviors. Motivated by these findings, we formulate data allocation as a scaling-law optimization problem. The derived allocations reveal an interpretable hub-and-island structure: highly transferable domains emerge as hubs that benefit many others and deserve strategic allocation, while isolated domains act as islands requiring direct investment. Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58% and generalizes well to unseen budgets with r=0.989. Downstream validation on disease classification and organ/lesion segmentation further confirms that the derived transfer-aware mixtures provide stronger pretrained representations for clinical 3D medical imaging tasks.

Related papers

• A Robust Unsupervised Domain Adaptation Framework for Medical Image Classification Using RKHS-MMD [0.9634859579172255]
  Labeling medical images is a major bottleneck in the field of medical imaging.<n>We propose an unsupervised domain adaptation framework that combines transfer learning with a Reproducing Kernel Space based Maximum Mean Discrepancy loss.<n>We show that RKHS-MMD performs better than the standard Maximum Mean Discrepancy in reducing modality gap.
  arXiv  Detail & Related papers  (2026-05-05T14:14:25Z)
• Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging? [2.0202525145391093]
  We propose a diffusion model (DM) trained on a source domain that generates synthetic cardiac MR images that resemble a given reference.<n>The synthetic data maintains spatial and structural fidelity, ensuring similarity to the source domain and compatibility with the segmentation mask.<n>We explore domain generalisation, where, domain-invariant segmentation models are trained on synthetic source domain data, and domain adaptation, where, we shift target domain data towards the source domain using the DM.
  arXiv  Detail & Related papers  (2025-08-08T13:57:48Z)
• Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation [56.52520416420957]
  We propose Multimodal Causal-Driven Representation Learning (MCDRL) to tackle domain generalization in medical image segmentation.<n>MCDRL consistently outperforms competing methods, yielding superior segmentation accuracy and exhibiting robust generalizability.
  arXiv  Detail & Related papers  (2025-08-07T03:41:41Z)
• M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation [4.327603420628437]
  Domain shift presents a significant challenge in applying Deep Learning to the segmentation of 3D medical images from sources like MRI and CT.<n>Within the proposed benchmark, we evaluate more than ten existing domain adaptation methods.<n>Our results show that none of them can consistently close the performance gap between the domains.
  arXiv  Detail & Related papers  (2025-02-24T10:29:32Z)
• Domain Generalization with Adversarial Intensity Attack for Medical Image Segmentation [27.49427483473792]
  In real-world scenarios, it is common for models to encounter data from new and different domains to which they were not exposed to during training. domain generalization (DG) is a promising direction as it enables models to handle data from previously unseen domains. We introduce a novel DG method called Adversarial Intensity Attack (AdverIN), which leverages adversarial training to generate training data with an infinite number of styles.
  arXiv  Detail & Related papers  (2023-04-05T19:40:51Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation [61.01704175938995]
  In this paper, we propose a novel Bidirectional Global-to-Local (BiGL) adaptation framework under a UDA scheme. Specifically, a bidirectional image synthesis and segmentation module is proposed to segment the brain tumor. The proposed method outperforms several state-of-the-art unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2021-05-17T10:11:45Z)
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation [9.659642285903418]
  Cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups.
  arXiv  Detail & Related papers  (2021-03-05T16:22:31Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)
• Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization [59.5104563755095]
  We introduce a simple but effective approach to improve the generalization capability of deep neural networks in the field of medical imaging classification. Motivated by the observation that the domain variability of the medical images is to some extent compact, we propose to learn a representative feature space through variational encoding.
  arXiv  Detail & Related papers  (2020-09-27T12:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.