Transforming Multimodal Models into Action Models for Radiotherapy

• URL: http://arxiv.org/abs/2502.04408v1
• Date: Thu, 06 Feb 2025 09:51:28 GMT
• Title: Transforming Multimodal Models into Action Models for Radiotherapy
• Authors: Matteo Ferrante, Alessandra Carosi, Rolando Maria D Angelillo, Nicola Toschi,
• Abstract summary: Radiotherapy a crucial cancer treatment demands precise planning to balance tumor preservation and eradication of healthy tissue. Traditional treatment planning (TP) is iterative, time-consuming, and reliant on human expertise. We propose a novel framework to transform a multimodal foundation model (MLM) into an action model for using a few-shot reinforcement learning approach.
• Score: 39.682133213072554
• License:
• Abstract: Radiotherapy is a crucial cancer treatment that demands precise planning to balance tumor eradication and preservation of healthy tissue. Traditional treatment planning (TP) is iterative, time-consuming, and reliant on human expertise, which can potentially introduce variability and inefficiency. We propose a novel framework to transform a large multimodal foundation model (MLM) into an action model for TP using a few-shot reinforcement learning (RL) approach. Our method leverages the MLM's extensive pre-existing knowledge of physics, radiation, and anatomy, enhancing it through a few-shot learning process. This allows the model to iteratively improve treatment plans using a Monte Carlo simulator. Our results demonstrate that this method outperforms conventional RL-based approaches in both quality and efficiency, achieving higher reward scores and more optimal dose distributions in simulations on prostate cancer data. This proof-of-concept suggests a promising direction for integrating advanced AI models into clinical workflows, potentially enhancing the speed, quality, and standardization of radiotherapy treatment planning.

Related papers

• Efficient Deep Learning-based Forward Solvers for Brain Tumor Growth Models [9.509686888976905]
  Glioblastoma, a highly aggressive brain tumor, poses major challenges due to its poor prognosis and high morbidity rates. Partial differential equation-based models offer promising potential to enhance therapeutic outcomes. We introduce an approach leveraging a neural forward solver with gradient-based optimization to significantly reduce calibration time.
  arXiv  Detail & Related papers  (2025-01-14T16:10:25Z)
• Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review [63.31328039424469]
  This tutorial provides a comprehensive survey of methods for fine-tuning diffusion models to optimize downstream reward functions. We explain the application of various RL algorithms, including PPO, differentiable optimization, reward-weighted MLE, value-weighted sampling, and path consistency learning.
  arXiv  Detail & Related papers  (2024-07-18T17:35:32Z)
• Diffusion Schrödinger Bridge Models for High-Quality MR-to-CT Synthesis for Head and Neck Proton Treatment Planning [9.599774878892665]
  Diffusion Schr"odinger Bridge Models (DSBM) is an innovative approach for high-quality MR-to-CT synthesis. Our research introduces DSBM, an innovative approach for high-quality MR-to-CT synthesis.
  arXiv  Detail & Related papers  (2024-04-17T20:48:19Z)
• A Learnable Prior Improves Inverse Tumor Growth Modeling [8.87818392404259]
  We propose a novel framework that leverages the unique strengths of both approaches in a synergistic manner. We showcase the effectiveness of integrating a rapid deep-learning algorithm with a high-precision evolution strategy in estimating brain tumor cell concentrations from magnetic resonance images.
  arXiv  Detail & Related papers  (2024-03-07T13:59:34Z)
• End-to-End Breast Cancer Radiotherapy Planning via LMMs with Consistency Embedding [47.360760580820966]
  We present RO-LMM, a comprehensive large multimodal model (LMM) tailored for the field of radiation oncology. This model effectively manages a series of tasks within the clinical workflow, including clinical context summarization, radiation treatment plan suggestion, and plan-guided target volume segmentation. We present a novel Consistency Embedding Fine-Tuning (CEFTune) technique, which boosts LMM's robustness to noisy inputs while preserving the consistency of handling clean inputs.
  arXiv  Detail & Related papers  (2023-11-27T14:49:06Z)
• Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [64.95663299945171]
  Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming. There exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. We propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs.
  arXiv  Detail & Related papers  (2023-09-10T22:05:24Z)
• A vision transformer-based framework for knowledge transfer from multi-modal to mono-modal lymphoma subtyping models [0.0]
  Whole Slide Image (WSI) analysis using deep learning models has shown promising potential for cancer diagnosis. We propose a vision transformer-based framework for distinguishing DLBCL cancer subtypes from high-resolution WSIs. Our experimental study conducted on a lymphoma dataset of 157 patients shows the promising performance of our monomodal classification model.
  arXiv  Detail & Related papers  (2023-08-02T17:05:36Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Utilizing Differential Evolution into optimizing targeted cancer treatments [0.0]
  Investigation of Differential Evolution was motivated by the high efficiency of variations of this technique in real-valued problems. A basic DE algorithm, namely "DE/rand/1" was used to optimize the simulated design of a targeted drug delivery system for tumor treatment on PhysiCell simulator.
  arXiv  Detail & Related papers  (2020-03-21T10:20:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.