TriPAH: Imbalance-Aware Tri-Prompt Affinity Hashing for Cross-Modal Medical Retrieval

• URL: http://arxiv.org/abs/2606.27010v1
• Date: Thu, 25 Jun 2026 13:24:34 GMT
• Title: TriPAH: Imbalance-Aware Tri-Prompt Affinity Hashing for Cross-Modal Medical Retrieval
• Abstract summary: Cross-modal medical hashing retrieval aims to enable efficient image-text search.<n>Current methods suffer from semantic fragmentation due to noisy clinical language, long-tailed labels, and brittle quantization.<n>We propose TriPAH, a Tri-Prompt Affinity Hashing framework.
• Score: 25.829128063343433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the era of big medical data, efficient cross-modal retrieval is pivotal for evidence-based diagnosis and large-scale case management. Cross-modal medical hashing retrieval aims to enable efficient image-text search and support downstream tasks such as case-based reasoning and decision support by learning compact, semantically aligned binary codes. However, current methods suffer from semantic fragmentation due to noisy clinical language, long-tailed labels, and brittle quantization that weakens alignment. We propose TriPAH, a Tri-Prompt Affinity Hashing framework. TriPAH synthesizes ontology-grounded, patient-level prompts conditioned on normalized clinical cues to yield low-noise textual representations for initial alignment. A lightweight prompt-token mixer performs hierarchical, multi-granularity alignment and produces quantization-ready features under an asymmetric multi-task objective coupling multi-positive contrastive alignment, imbalance-aware classification, and progressive quantization regularization. A patient-level consistency module further stabilizes codes across complementary views. Extensive experiments on three public datasets demonstrate that TriPAH significantly outperforms state-of-the-art methods.

Related papers

• SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation [5.487826115955642]
  Vision-language pre-training serves as a cornerstone for medical multimodal representation learning.<n>We propose textbfSCALPEL, a textbfSemantic textbfCross-modal textbfAlignment framework via textbfLLM-textbfPowered textbfEncoder textbfLearning.
  arXiv  Detail & Related papers  (2026-07-29T13:12:32Z)
• Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs [53.697403481143404]
  Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks.<n>But they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback.
  arXiv  Detail & Related papers  (2026-06-10T18:35:36Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models [59.180043227905294]
  We propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework for efficient 3D medical image understanding.<n>We show that MedPruner enables models such as MedGemma to maintain or even exceed their original performance while retaining fewer than 5% of visual tokens.
  arXiv  Detail & Related papers  (2026-03-12T07:37:00Z)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding [66.78251988482222]
  Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning.<n>Current benchmarks for medical image understanding generally focus on the final answer while ignoring the reasoning path.<n>M3CoTBench aims to foster the development of transparent, trustworthy, and diagnostically accurate AI systems for healthcare.
  arXiv  Detail & Related papers  (2026-01-13T17:42:27Z)
• Staged Voxel-Level Deep Reinforcement Learning for 3D Medical Image Segmentation with Noisy Annotations [4.581671524490035]
  We propose an end-to-end Staged Voxel-Level Deep Reinforcement Learning framework for robust medical image segmentation under noisy annotations.<n>This framework employs a dynamic iterative update strategy to automatically mitigate the impact of erroneous labels without requiring manual intervention.
  arXiv  Detail & Related papers  (2026-01-07T12:39:54Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning [21.44813166265882]
  Large Language Models (LLMs) have shown strong performance on static medical Question Answering (QA) tasks.<n>This paper introduces TriMediQ, a triplet-structured approach that enhances the reasoning reliability of LLMs.<n> Experiments on two interactive medical QA benchmarks show that TriMediQ achieves up to 10.4% improvement in accuracy over five existing baselines.
  arXiv  Detail & Related papers  (2025-10-03T22:11:17Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• Meta-Entity Driven Triplet Mining for Aligning Medical Vision-Language Models [9.76070837929117]
  Existing alignment methods prioritize separation between disease classes over segregation of fine-grained pathology attributes.<n>Here, we propose MedTrim, a novel method that enhances image-text alignment through multimodal triplet learning.<n>Our demonstrations indicate that MedTrim improves performance in downstream retrieval and classification tasks compared to state-of-the-art alignment methods.
  arXiv  Detail & Related papers  (2025-04-22T14:17:51Z)
• SGTC: Semantic-Guided Triplet Co-training for Sparsely Annotated Semi-Supervised Medical Image Segmentation [12.168303995947795]
  We propose a novel Semantic-Guided Triplet Co-training framework.<n>It achieves high-end medical image segmentation by only annotating three slices of a few volumetric samples.<n>Our method outperforms most state-of-the-art semi-supervised counterparts under sparse annotation settings.
  arXiv  Detail & Related papers  (2024-12-20T03:31:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.