A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans

• URL: http://arxiv.org/abs/2608.21140v1
• Date: Fri, 21 Aug 2026 14:16:46 GMT
• Title: A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans
• Abstract summary: Modern vision-language models (VLMs) show promising performance on many medical imaging tasks.<n>We present a modular medical imaging agent for binary spatial relation verification in axial CT slices.<n>The best-performing hybrid configuration reaches 94.1% accuracy and 94.2% F1, outperforming direct Qwen2-VL prompting by 42.5 percentage points in accuracy.
• Score: 7.8047270656196766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable spatial understanding is an important prerequisite for future medical vision-language systems that aim to support radiological report generation and structured image understanding. While modern vision-language models (VLMs) show promising performance on many medical imaging tasks, recent evidence suggests they remain weak in controlled spatial reasoning and often fail to reliably ground spatial relations in image evidence. Given that radiological reasoning hinges on understanding the relative positions of anatomical structures and findings, this spatial weakness poses risks to diagnostic accuracy. We present a modular medical imaging agent for binary spatial relation verification in axial CT slices. Instead of directly predicting spatial answers end-to-end, the system decomposes the task into explicit stages: language parsing, anatomical localization, and deterministic geometric verification. Natural-language queries are converted into structured relation tuples, queried organs are localized with a YOLO-based detector, and the final spatial decision is computed from object centers using deterministic geometric rules. We evaluate the approach on the held-out MIRP spatial QA benchmark and compare it against representative end-to-end VLM baselines. The best-performing hybrid configuration reaches 94.1% accuracy and 94.2% F1, outperforming direct Qwen2-VL prompting by 42.5 percentage points in accuracy, while preserving interpretable intermediate representations and auditable reasoning stages. The results suggest that explicit modular spatial verification can serve as a promising building block for future report-oriented medical imaging agents.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Images [48.646448077284276]
  We propose Spatial-FAD, a framework for effective few-shot medical anomaly detection.<n>It improves lesion localization by combining VFM spatial priors with CLIP semantics.<n>Our method achieves an average improvement of over 11.4% in Dice score and 1.8% in AUC.
  arXiv  Detail & Related papers  (2026-09-11T05:21:01Z)
• CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement [3.586844533420584]
  We introduce CARE-X, a chest X-ray Vision-Language Model (VLM) that unifies auxiliary discriminative supervision with reward-aligned generation.<n> CARE-X augments its generative backbone with focal-loss classification and composite-loss grounding heads, co-trained alongside the language-modeling objective.<n>The result is state-of-the-art performance on the majority of metrics across four report-generation benchmarks, 94.0% VQA accuracy on ReXVQA, and generative spatial decoding that reaches near parity with dedicated detection heads.
  arXiv  Detail & Related papers  (2026-08-04T16:23:39Z)
• Lost in Volume: The CT-SpatialVQA Benchmark for Evaluating Semantic-Spatial Understanding of 3D Medical Vision-Language Models [3.6659638651482536]
  Recent advances in 3D medical vision-language models have enabled joint reasoning over volumetric images and text.<n>It remains unclear whether these models learn spatially grounded anatomy from 3D volumes or rely primarily on learned priors and language correlations.<n>We introduce CT-SpatialVQA, a benchmark designed to evaluate semantic-spatial reasoning in 3D CT data.
  arXiv  Detail & Related papers  (2026-05-09T08:16:00Z)
• MedScribe: Clinically Grounded CT Reporting through Agentic Workflows [13.40306812882295]
  Vision-language models (VLMs) have shown potential for automated radiology report generation.<n>We introduce MedScribe, a hypothesis-driven framework that reformulates report generation as an iterative evidence acquisition process.
  arXiv  Detail & Related papers  (2026-05-03T08:32:40Z)
• Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models [62.932580559941414]
  Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
  arXiv  Detail & Related papers  (2026-03-16T17:20:38Z)
• Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers [57.54843029965778]
  Controllable pathology image synthesis requires reliable regulation of spatial layout, tissue morphology, and semantic detail.<n>We propose In-Context Diffusion Transformer (IC-DiT), a layout-aware generative model that incorporates spatial layouts, textual descriptions, and visual embeddings into a unified diffusion transformer.<n>IC-DiT achieves higher fidelity, stronger spatial controllability, and better diagnostic consistency than existing methods.
  arXiv  Detail & Related papers  (2026-03-11T06:14:11Z)
• S-Chain: Structured Visual Chain-of-Thought For Medicine [81.97605645734741]
  We introduce S-Chain, the first large-scale dataset of 12,000 expert-annotated medical images with bounding boxes and structured visual CoT (SV-CoT)<n>The dataset further supports 16 languages, totaling over 700k VQA pairs for broad multilingual applicability.<n>S-Chain establishes a new benchmark for grounded medical reasoning and paves the way toward more trustworthy and explainable medical vision-language models.
  arXiv  Detail & Related papers  (2025-10-26T15:57:14Z)
• RAU: Reference-based Anatomical Understanding with Vision Language Models [26.06602931463068]
  We introduce RAU, a framework for reference-based anatomical understanding with vision-language models (VLMs)<n>We first show that a VLM learns to identify anatomical regions through relative spatial reasoning between reference and target images.<n>Next, we demonstrate that the VLM-derived spatial cues can be seamlessly integrated with the fine-grained segmentation capability of SAM2.
  arXiv  Detail & Related papers  (2025-09-26T14:32:03Z)
• PRS-Med: Position Reasoning Segmentation with Vision-Language Model in Medical Imaging [6.411386758550256]
  PRS-Med is a framework that integrates vision-language models with segmentation capabilities to generate both accurate segmentation masks and corresponding spatial reasoning outputs.<n> MMRS dataset provides diverse, spatially-grounded question-answer pairs to address the lack of position reasoning data in medical imaging.
  arXiv  Detail & Related papers  (2025-05-17T06:42:28Z)
• Uncertainty-aware Medical Diagnostic Phrase Identification and Grounding [72.18719355481052]
  We introduce a novel task called Medical Report Grounding (MRG)<n>MRG aims to directly identify diagnostic phrases and their corresponding grounding boxes from medical reports in an end-to-end manner.<n>We propose uMedGround, a robust and reliable framework that leverages a multimodal large language model to predict diagnostic phrases.
  arXiv  Detail & Related papers  (2024-04-10T07:41:35Z)
• Cross-Modal Causal Intervention for Medical Report Generation [107.76649943399168]
  Radiology Report Generation (RRG) is essential for computer-aided diagnosis and medication guidance.<n> generating accurate lesion descriptions remains challenging due to spurious correlations from visual-linguistic biases.<n>We propose a two-stage framework named CrossModal Causal Representation Learning (CMCRL)<n> Experiments on IU-Xray and MIMIC-CXR show that our CMCRL pipeline significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.