RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning

• URL: http://arxiv.org/abs/2606.28392v1
• Date: Tue, 23 Jun 2026 18:05:55 GMT
• Title: RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning
• Abstract summary: We present RADIANT-PET, a reasoning-augmented framework that couples a high-sensitivity voxel-level segmentation model with lesion-level large language model (LLM) adjudication.<n>Across AutoPET and an OSU test cohort, RADIANT-PET consistently outperforms strong image-only baselines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate lesion segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT-PET, a reasoning-augmented framework that couples a high-sensitivity voxel-level segmentation model with lesion-level large language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion-level reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, rewarding correct lesion classification and anatomically concordant site assignment. Across AutoPET and an OSU test cohort, RADIANT-PET consistently outperforms strong image-only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM-based lesion-level reasoning adds a novel reasoning layer beyond conventional segmentation, suppressing physiologic false positives and aligning voxel-level predictions with clinical interpretation. The project repository is available at: https://github.com/jwang-580/RADIANT-PET.

Related papers

• MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation [2.779944317864993]
  MUST-PET is a multimodal, multi-tracer SSL framework for generalizable whole-body PET-CT lesion segmentation.<n>It reduces reconstruction error, improves lesion segmentation over training from scratch, and performs well with limited labeled data.
  arXiv  Detail & Related papers  (2026-08-20T06:02:54Z)
• Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided [62.10276009853225]
  Reliable pelvic bone segmentation from CT is essential for robot-assisted pelvic trauma surgery.<n>Existing methods show limited effectiveness for PBS.<n>We propose a novel closed-loop dynamic Reliability-Guided framework (ReGA) for PBS.
  arXiv  Detail & Related papers  (2026-08-01T08:13:20Z)
• Weakly Supervised Pathology-Informed Representation Learning for PET-Based Content Retrieval of Intra-Tumour Heterogeneity [0.16999906736255554]
  The proposed method was designed to use H&E derived information during training while maintaining PET only inference.<n>A teacher student training strategy was used to learn the PET tumour derived voxel representations.
  arXiv  Detail & Related papers  (2026-07-21T06:40:14Z)
• Improving PET/CT-Based Whole-Body Lesion Segmentation Using Prediction Uncertainty-Augmented Models [4.533732724414947]
  Lesion delineation from PET/CT imaging is clinically challenging due to subtle imaging features, confounders, and inter-reader variability.<n>Existing deep learning approaches suffer from training-relatedity, inconsistent predictions, missed lesions in high tumor-burden cases, and lack uncertainty.<n>We propose an uncertainty-aware framework for whole-body PET/CT lesion segmentation.
  arXiv  Detail & Related papers  (2026-06-08T19:49:23Z)
• An Open Multi-Center Whole-Body FDG PET/CT Foundation Model for Tumor Segmentation [9.922425508121012]
  We present an open-source, multi-center, whole-body FDG PET/CT foundation model utilizing 4,997 harmonized scans from four public datasets.<n>Our framework employs hierarchical UNet-shaped backbones with early channel-wise concatenation, enabling anatomical and metabolic features to interact.<n>It provides a robust, open-source basis for advancing automated oncologic imaging.
  arXiv  Detail & Related papers  (2026-05-20T23:59:50Z)
• OmniCT: Towards a Unified Slice-Volume LVLM for Comprehensive CT Analysis [53.01523944168442]
  Clinical interpretation relies on both slice-driven local features and volume-driven spatial representations.<n>Existing Large Vision-Language Models (LVLMs) remain fragmented in CT slice versus volumetric understanding.<n>We present OmniCT, a powerful unified slice-volume LVLM for CT scenarios.
  arXiv  Detail & Related papers  (2026-02-18T00:42:41Z)
• A Generalist Foundation Model for Total-body PET/CT Enables Diagnostic Reporting and System-wide Metabolic Profiling [17.709593152803695]
  SDF-HOLO is a multimodal foundation model for holistic total-body PET/CT, pre-trained on more than 10,000 patients.<n>It decouples CT and PET representation learning with dual-stream encoders and couples them through a cross-modal interaction module.<n>The model enables system-wide metabolic profiling and reveals tumor-associated fingerprints of inter-organ metabolic network interactions.
  arXiv  Detail & Related papers  (2026-01-19T08:30:09Z)
• Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench [48.60251555171943]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in tasks such as abnormality detection and report generation for anatomical modalities.<n>In this work, we quantify a fundamental functional perception gap: the inability of current vision encoders to decode functional tracer biodistribution independent of morphological priors.<n>We introduce PET-Bench, the first large-scale functional imaging benchmark comprising 52,308 hierarchical QA pairs from 9,732 multi-site, multi-tracer PET studies.<n>Our results demonstrate that AVA effectively bridges the perception gap, transforming CoT from a source of hallucination into a robust inference tool and improving diagnostic
  arXiv  Detail & Related papers  (2026-01-06T05:58:50Z)
• Vision-Language Models for Automated 3D PET/CT Report Generation [13.781844347232079]
  AutoPET/CT report generation is increasingly important for reducing clinical workload.<n>PETRG-3D is an end-to-end 3D dual-branch framework that encodes PET and CT volumes and incorporates style-adaptive prompts.<n>PETRG-Score is a lymphoma-specific evaluation protocol that measures metabolic and structural findings across curated anatomical regions.
  arXiv  Detail & Related papers  (2025-11-25T10:07:57Z)
• FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence [46.71469172542448]
  This paper presents FactPICO, a factuality benchmark for plain language summarization of medical texts. It consists of 345 plain language summaries of abstracts generated from three randomized controlled trials (RCTs) We assess the factuality of critical elements of RCTs in those summaries, as well as the reported findings concerning these.
  arXiv  Detail & Related papers  (2024-02-18T04:45:01Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• Evaluating Large Language Models for Radiology Natural Language Processing [68.98847776913381]
  The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP) This study seeks to bridge this gap by critically evaluating thirty two LLMs in interpreting radiology reports.
  arXiv  Detail & Related papers  (2023-07-25T17:57:18Z)
• Whole-Body Lesion Segmentation in 18F-FDG PET/CT [11.662584140924725]
  The proposed model is designed on the basis of the joint 2D and 3D nnUNET architecture to predict lesions across the whole body. We evaluate the proposed method in the context of AutoPet Challenge, which measures the lesion segmentation performance in the metrics of dice score, false-positive volume and false-negative volume.
  arXiv  Detail & Related papers  (2022-09-16T10:49:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.