Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer

• URL: http://arxiv.org/abs/2605.07561v1
• Date: Fri, 08 May 2026 10:34:27 GMT
• Title: Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer
• Abstract summary: Pathological complete response (pCR) is a key prognostic factor in breast cancer patients undergoing neoadjuvant therapy.<n>We propose a clinically-guided attention learning framework for pCR prediction from breast magnetic resonance imaging (MRI)<n>The framework is evaluated through external validation across heterogeneous MRI cohorts.
• Score: 4.6719795751452295
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Pathological complete response (pCR) is a key prognostic factor in breast cancer patients undergoing neoadjuvant therapy, strongly associated with long-term survival and treatment personalization. However, accurate pre-treatment pCR prediction remains challenging due to severe class imbalance and limited generalizability across diverse clinical settings. In this work, we propose a multimodal stepwise clinically-guided attention learning framework for pCR prediction from breast magnetic resonance imaging (MRI), designed to address these limitations through medically grounded spatial guidance and multimodal integration. The approach follows a stepwise training strategy inspired by physician reasoning: the model first learns global discriminative imaging patterns, then attention mechanisms are introduced to constrain the network toward tumor regions, and finally clinical variables are integrated to refine decision-making. This guidance strategy encourages prioritization of task-relevant features, improving identification of responders despite their limited representation in the dataset. Moreover, grounding attention in anatomically consistent tumor regions reduces reliance on dataset-specific patterns, thereby enhancing cross-institutional generalization. The framework is evaluated through external validation across heterogeneous MRI cohorts. Compared to non-guided single-stage baselines, the proposed approach improves sensitivity while maintaining competitive specificity, and produces anatomically coherent attention maps that support interpretation of the model's predictions. These findings highlight the potential of clinically-guided multimodal attention learning for robust and generalizable pCR prediction in breast cancer.

Related papers

• ClinRAG-GRAPH: Clinical-prior Retrieval-Augmented Graph Model with Domain Adversarial Learning for Breast pCR Prediction [27.661246388005384]
  We propose ClinRAG-GRAPH, a framework for pre-treatment pathological complete response (pCR) prediction.<n>ClinRAG-GRAPH constructs an intra-patient clinical-prior graph and applies a prior-guided relation-aware graph convolutional network for structured multimodal representation learning.<n>ClinRAG-GRAPH achieves AUCs of 0.815 on the internal test set and 0.774/0.712 on two external test sets, demonstrating robust pre-treatment pCR prediction across centers.
  arXiv  Detail & Related papers  (2026-07-01T11:28:01Z)
• Multi-Modal Machine Learning for Breast Cancer Recurrence Prediction [13.264276237086731]
  Breast cancer recurrence is a leading cause of long-term mortality among survivors.<n>Traditional predictive models, often limited to either structured or unstructured data alone, struggle to capture the full clinical context.<n>This study examines the impact of integrating multi-modal clinical data, including treatment records, pathology reports, and clinician notes, on recurrence prediction.
  arXiv  Detail & Related papers  (2026-06-01T21:06:07Z)
• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• Learning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC [1.3160173536707265]
  Major pathological response (pR) following neoadjuvant therapy is a clinically meaningful endpoint in non-small cell lung cancer.<n>We propose a multimodal deep learning framework designed to address these constraints by integrating foundation model-based CT feature extraction with a missing-aware architecture for clinical variables.<n>A weighted fusion mechanism is employed to leverage the complementary contributions of imaging and clinical modalities, yielding a multimodal model that consistently outperforms both unimodal imaging and clinical baselines.
  arXiv  Detail & Related papers  (2026-03-16T10:51:52Z)
• MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement [63.82954136824963]
  Medical Vision-Language Models excel at perception tasks with complex clinical reasoning required in real-world scenarios.<n>We propose a novel reasoning MedVLM that addresses these challenges through domain-specific adaptation and guideline reinforcement.
  arXiv  Detail & Related papers  (2026-01-16T02:32:07Z)
• Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer [0.7003240413492382]
  This study proposes a novel approach combining Multimodal Deep Learning with intrinsic eXplainable Artificial Intelligence techniques to predict pathological response in non-small cell lung cancer patients undergoing neoadjuvant therapy.<n>Due to the limitations of existing radiomics and unimodal deep learning approaches, we introduce an intermediate fusion strategy that integrates imaging and clinical data, enabling efficient interaction between data modalities.<n>Results demonstrate improved predictive accuracy and explainability, providing insights into optimal data integration strategies for clinical applications.
  arXiv  Detail & Related papers  (2025-05-02T16:57:37Z)
• Doctor-in-the-Loop: An Explainable, Multi-View Deep Learning Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer [0.6800826356148091]
  Non-small cell lung cancer (NSCLC) remains a major global health challenge.<n>We propose Doctor-in-the-Loop, a novel framework that integrates expert-driven domain knowledge with explainable artificial intelligence techniques.<n>Our approach employs a gradual multi-view strategy, progressively refining the model's focus from broad contextual features to finer, lesion-specific details.
  arXiv  Detail & Related papers  (2025-02-21T16:35:30Z)
• TopoTxR: A topology-guided deep convolutional network for breast parenchyma learning on DCE-MRIs [49.69047720285225]
  We propose a novel topological approach that explicitly extracts multi-scale topological structures to better approximate breast parenchymal structures. We empirically validate emphTopoTxR using the VICTRE phantom breast dataset. Our qualitative and quantitative analyses suggest differential topological behavior of breast tissue in treatment-na"ive imaging.
  arXiv  Detail & Related papers  (2024-11-05T19:35:10Z)
• Radiomics-guided Multimodal Self-attention Network for Predicting Pathological Complete Response in Breast MRI [3.6852491526879687]
  This study presents a model that predicts pCR in breast cancer patients using dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) and apparent diffusion coefficient (ADC) maps. Our approach extracts features from both DCE MRI and ADC using an encoder with a self-attention mechanism, leveraging radiomics to guide feature extraction from tumor-related regions.
  arXiv  Detail & Related papers  (2024-06-05T04:49:55Z)
• Enhancing Clinical Support for Breast Cancer with Deep Learning Models using Synthetic Correlated Diffusion Imaging [66.63200823918429]
  We investigate enhancing clinical support for breast cancer with deep learning models. We leverage a volumetric convolutional neural network to learn deep radiomic features from a pre-treatment cohort. We find that the proposed approach can achieve better performance for both grade and post-treatment response prediction.
  arXiv  Detail & Related papers  (2022-11-10T03:02:12Z)
• Exploiting segmentation labels and representation learning to forecast therapy response of PDAC patients [60.78505216352878]
  We propose a hybrid deep neural network pipeline to predict tumour response to initial chemotherapy. We leverage a combination of representation transfer from segmentation to classification, as well as localisation and representation learning. Our approach yields a remarkably data-efficient method able to predict treatment response with a ROC-AUC of 63.7% using only 477 datasets in total.
  arXiv  Detail & Related papers  (2022-11-08T11:50:31Z)
• Adaptive unsupervised learning with enhanced feature representation for intra-tumor partitioning and survival prediction for glioblastoma [12.36330256366686]
  We propose an adaptive unsupervised learning approach for efficient MRI intra-tumor partitioning and glioblastoma survival prediction. A novel and problem-specific Feature-enhanced Auto-Encoder (FAE) is developed to enhance the representation of pairwise clinical modalities. The results demonstrate that the proposed approach can produce robust and clinically relevant MRI sub-regions and statistically significant survival predictions.
  arXiv  Detail & Related papers  (2021-08-21T02:47:59Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.