Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation

• URL: http://arxiv.org/abs/2603.08305v1
• Date: Mon, 09 Mar 2026 12:27:17 GMT
• Title: Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation
• Abstract summary: We propose a retrieval-augmented approach for Text-to-CT generation that integrates semantic and anatomical information under a realistic inference setting.<n>Experiments on the CT-RATE dataset show that retrieval-augmented generation improves image fidelity and clinical consistency compared to text-only baselines.<n>This work introduces a principled and scalable mechanism to bridge semantic conditioning and anatomical plausibility in volumetric medical image synthesis.
• Score: 1.5532758127091075
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Text-conditioned generative models for volumetric medical imaging provide semantic control but lack explicit anatomical guidance, often resulting in outputs that are spatially ambiguous or anatomically inconsistent. In contrast, structure-driven methods ensure strong anatomical consistency but typically assume access to ground-truth annotations, which are unavailable when the target image is to be synthesized. We propose a retrieval-augmented approach for Text-to-CT generation that integrates semantic and anatomical information under a realistic inference setting. Given a radiology report, our method retrieves a semantically related clinical case using a 3D vision-language encoder and leverages its associated anatomical annotation as a structural proxy. This proxy is injected into a text-conditioned latent diffusion model via a ControlNet branch, providing coarse anatomical guidance while maintaining semantic flexibility. Experiments on the CT-RATE dataset show that retrieval-augmented generation improves image fidelity and clinical consistency compared to text-only baselines, while additionally enabling explicit spatial controllability, a capability inherently absent in such approaches. Further analysis highlights the importance of retrieval quality, with semantically aligned proxies yielding consistent gains across all evaluation axes. This work introduces a principled and scalable mechanism to bridge semantic conditioning and anatomical plausibility in volumetric medical image synthesis. Code will be released.

Related papers

• Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning [80.18561652862725]
  Generalizing across subjects remains challenging in invasive neural recordings.<n>We propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space.
  arXiv  Detail & Related papers  (2026-07-03T03:59:16Z)
• ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training [26.518178155362634]
  We propose a principled vision-language pre-training framework for fine-grained medical volumetric representation learning.<n>ASAP integrates three key components: (1) an anatomy-aware knowledge injection module that incorporates organ-level structural priors; (2) a semantically-adaptive selective alignment mechanism that associates sentence-level findings with localized regions; and (3) a semantically-adaptive fusion module for effective interaction between anatomically informed visual features and grounded textual cues.
  arXiv  Detail & Related papers  (2026-05-30T07:59:21Z)
• Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation [29.61647712180762]
  Self-supervised pre-training paradigm has gained increasing prominence for learning transferable representations in medical imaging.<n>We propose an anatomy-anchored ultrasound self-supervision framework ANAUS that shifts representation learning from generic visual regions to clinically meaningful anatomical structures.
  arXiv  Detail & Related papers  (2026-05-25T03:52:58Z)
• GLeVE: Graph-Guided Lesion Grounding with Proposal Verification in 3D CT [16.979389238358937]
  Grounding radiology report descriptions to 3D CT volumes is essential for verifiable clinical interpretation.<n>Existing report-assisted and vision-language grounding methods rely on phrase-level alignment or dense pixel supervision.<n>We propose GLeVE, a graph-guided lesion grounding framework with anatomical prior verification and octree-based autoregressive refinement.
  arXiv  Detail & Related papers  (2026-05-21T15:30:41Z)
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• Fake It Right: Injecting Anatomical Logic into Synthetic Supervised Pre-training for Medical Segmentation [21.75204301463342]
  Vision Transformers (ViTs) excel in 3D medical segmentation but require massive datasets.<n>Formula-Driven Supervised Learning (F) offers a privacy-preserving alternative by pre-training on synthetic mathematical primitives.<n>We propose an annotated-Informed Synthetic Anatomy Supervised Pre-training framework unifying F's infinite scalability with anatomical realism.
  arXiv  Detail & Related papers  (2026-03-01T08:15:18Z)
• Rescind: Countering Image Misconduct in Biomedical Publications with Vision-Language and State-Space Modeling [8.024142807011378]
  We present the first vision-language guided framework for both generating and detecting biomedical image forgeries.<n>By combining diffusion-based synthesis with vision-language prompting, our method enables realistic and semantically controlled manipulations.<n>Integscan achieves state of the art performance in both detection and localization, establishing a strong foundation for automated scientific integrity analysis.
  arXiv  Detail & Related papers  (2026-01-12T22:13:58Z)
• Semantic Visual Anomaly Detection and Reasoning in AI-Generated Images [96.43608872116347]
  AnomReason is a large-scale benchmark with structured annotations as quadruple textbfAnomAgent<n>AnomReason and AnomAgent serve as a foundation for measuring and improving the semantic plausibility of AI-generated images.
  arXiv  Detail & Related papers  (2025-10-11T14:09:24Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• GRASPing Anatomy to Improve Pathology Segmentation [67.98147643529309]
  We introduce GRASP, a modular plug-and-play framework that enhances pathology segmentation models.<n>We evaluate GRASP on two PET/CT datasets, conduct systematic ablation studies, and investigate the framework's inner workings.
  arXiv  Detail & Related papers  (2025-08-05T12:26:36Z)
• Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation [52.51005875755718]
  We focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse.<n>Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings.<n>Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences.
  arXiv  Detail & Related papers  (2025-05-21T05:29:55Z)
• Anatomical Attention Alignment representation for Radiology Report Generation [10.887845017050477]
  Anatomical Attention Alignment Network (A3Net) is a framework that enhance visual-textual understanding by constructing hyper-visual representations.<n>Our approach integrates a knowledge dictionary of anatomical structures with patch-level visual features, enabling the model to effectively associate image regions with their corresponding anatomical entities.<n> Experimental results on IU X-Ray and MIMIC-CXR datasets demonstrate that A3Net significantly improves both visual perception and text generation quality.
  arXiv  Detail & Related papers  (2025-05-12T15:54:50Z)
• Segment as You Wish -- Free-Form Language-Based Segmentation for Medical Images [30.673958586581904]
  We introduce FLanS, a novel medical image segmentation model that handles various free-form text prompts.<n> FLanS is trained on a large-scale dataset of over 100k medical images from 7 public datasets.
  arXiv  Detail & Related papers  (2024-10-02T16:34:32Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.