Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis

• URL: http://arxiv.org/abs/2608.03225v1
• Date: Tue, 04 Aug 2026 06:58:26 GMT
• Title: Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis
• Abstract summary: Human-interpretable computer-aided diagnosis is crucial for clinical decision making.<n>Applying concept-based models across diverse modalities is challenging due to heterogeneous concept varying in availability, granularity, and semantics across cohorts.<n>We propose UniCon, an open-linguistic unified concept learning framework for multimodal interpretable vision-language diagnosis.
• Score: 27.376170530265295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-interpretable computer-aided diagnosis is crucial for clinical decision making. Concept-based models excel by providing transparent reasoning and enabling post-hoc, clinician-in-the-loop interventions. However, their rigid dataset-specific adaptation inherently restricts cross-site generalization. Applying them across diverse modalities, such as dermoscopic and clinical photographs, is challenging due to heterogeneous concept taxonomies varying in availability, granularity, and semantics across cohorts. Consequently, adapting Foundation Vision-Language Models (FVLMs) demands costly label engineering and repeated post-training. Existing intervention mechanisms remain rigidly tied to predefined concepts, lacking adaptability and hindering scalable dermatology CAD deployment. To address these bottlenecks, we propose UniCon, an open-linguistic unified concept learning framework for multimodal interpretable vision-language diagnosis. UniCon resolves these challenges through three contributions: (1) A shared semantic representation space via a unified concept prototype codebook, seamlessly coordinating heterogeneous concept systems across modalities without dataset-specific retraining. (2) Open-linguistic based multi-faceted semantic specifications to overcome sparse textual label limitations, improving boundary sensitivity in uncertain clinical contexts. (3) A robust, cross-site adjustable intervention interface powered by reliability-gated bottleneck aggregation, enabling consistent reasoning and transferable clinician corrections. Extensive experiments demonstrate that beyond securing top-tier diagnostic accuracy, UniCon successfully bridges disparate clinical taxonomies, unlocking unprecedented cross-site intervention capabilities. Code is available at https://github.com/wuchengyu123/UniCon.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• TRACE: Training-time Report-guided and Clinically Ordered Concept Editing [9.375094449682043]
  We propose Training-time Report-guided and Clinically Ordered Concept Editing (TRACE)<n>TRACE refines image-derived concepts through a teacher-guided editing mechanism within a malignancy-aware ordered concept space.<n>We introduce Strategic Concept Missing Training (SCMT) and train an image-only self-editor via edit distillation for autonomous concept refinement.
  arXiv  Detail & Related papers  (2026-08-21T07:26:39Z)
• ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology [33.27476666910628]
  ConceptM$3$oE embeds concept formation directly within interaction-aware mixture-of-experts pathways.<n>The architecture decomposes evidence into modality-specific, redundant, and synergistic experts, which are then projected into structured concept bottlenecks.<n>In data-limited regimes, ConceptM$3$oE improves limited-data performance, increasing macro-F1 from 56.41% to 66.70% at small training sizes.
  arXiv  Detail & Related papers  (2026-05-23T04:55:16Z)
• DCG-Net: Dual Cross-Attention with Concept-Value Graph Reasoning for Interpretable Medical Diagnosis [14.035057926904559]
  Concept Bottleneck Models (CBMs) structure predictions through human-interpretable clinical concepts.<n>Existing CBMs typically overlook the contextual dependencies among concepts.<n>We propose an end-to-end interpretable framework emphDCG-Net that integrates multimodal alignment with structured concept reasoning.
  arXiv  Detail & Related papers  (2026-03-20T01:11:58Z)
• TGC-Net: A Structure-Aware and Semantically-Aligned Framework for Text-Guided Medical Image Segmentation [56.09179939570486]
  We propose TGC-Net, a CLIP-based framework focusing on parameter-efficient, task-specific adaptations.<n>TGC-Net achieves state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-12-24T12:06:26Z)
• MIRNet: Integrating Constrained Graph-Based Reasoning with Pre-training for Diagnostic Medical Imaging [67.74482877175797]
  MIRNet is a novel framework that integrates self-supervised pre-training with constrained graph-based reasoning.<n>We introduce TongueAtlas-4K, a benchmark comprising 4,000 images annotated with 22 diagnostic labels.
  arXiv  Detail & Related papers  (2025-11-13T06:30:41Z)
• Interpretable Neuropsychiatric Diagnosis via Concept-Guided Graph Neural Networks [56.75602443936853]
  One in five adolescents currently live with a diagnosed mental or behavioral health condition, such as anxiety, depression, or conduct disorder.<n>While prior works use graph neural network (GNN) approaches for disorder prediction, they remain black-boxes, limiting their reliability and clinical translation.<n>In this work, we propose a concept-based diagnosis framework that that encodes interpretable functional connectivity concepts.<n>Our design ensures predictions through clinically meaningful connectivity patterns, enabling both interpretability and strong predictive performance.
  arXiv  Detail & Related papers  (2025-10-02T19:38:46Z)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis [56.373297358647655]
  Retrieval-Augmented Diagnosis (RAD) is a novel framework that injects external knowledge into multimodal models directly on downstream tasks.<n>RAD operates through three key mechanisms: retrieval and refinement of disease-centered knowledge from multiple medical sources, a guideline-enhanced contrastive loss transformer, and a dual decoder.
  arXiv  Detail & Related papers  (2025-09-24T10:36:14Z)
• Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays [46.667021835430155]
  We map clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs)<n>The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features.<n>Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging.
  arXiv  Detail & Related papers  (2025-06-04T15:23:12Z)
• A Two-Step Concept-Based Approach for Enhanced Interpretability and Trust in Skin Lesion Diagnosis [6.6635650150737815]
  Concept Bottleneck Models (CBMs) offer inherent interpretability by constraining the final disease prediction on a set of human-understandable concepts.<n>We introduce a novel two-step methodology that addresses both of these challenges.<n>By simulating the two stages of a CBM, we utilize a pretrained Vision Language Model (VLM) to automatically predict clinical concepts, and an off-the-shelf Large Language Model (LLM) to generate disease diagnoses.
  arXiv  Detail & Related papers  (2024-11-08T14:52:42Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.