Leveraging the Structure of Medical Data for Improved Representation Learning

• URL: http://arxiv.org/abs/2507.02987v3
• Date: Thu, 24 Jul 2025 12:44:31 GMT
• Title: Leveraging the Structure of Medical Data for Improved Representation Learning
• Abstract summary: Building generalizable medical AI systems requires pretraining strategies that are data-efficient and domain-aware.<n>We propose a self-supervised framework that leverages the inherent structure of medical datasets.<n>We show strong performance compared to supervised objectives and baselines being trained without leveraging structure.
• Score: 12.175375511821352
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Building generalizable medical AI systems requires pretraining strategies that are data-efficient and domain-aware. Unlike internet-scale corpora, clinical datasets such as MIMIC-CXR offer limited image counts and scarce annotations, but exhibit rich internal structure through multi-view imaging. We propose a self-supervised framework that leverages the inherent structure of medical datasets. Specifically, we treat paired chest X-rays (i.e., frontal and lateral views) as natural positive pairs, learning to reconstruct each view from sparse patches while aligning their latent embeddings. Our method requires no textual supervision and produces informative representations. Evaluated on MIMIC-CXR, we show strong performance compared to supervised objectives and baselines being trained without leveraging structure. This work provides a lightweight, modality-agnostic blueprint for domain-specific pretraining where data is structured but scarce

Related papers

• EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment [15.371433625713232]
  EndoVLM is a novel vision-language FM pre-trained on over 348K endoscopic examinations.<n>An Anatomy-Guided Sparse Pooling mechanism utilizes textual descriptions as queries to drive sparse attention.<n>A Progressive Semantic-Aware Alignment strategy models clinical taxonomy (anatomy and pathological status) via structured soft targets.<n>A Semantic-Concentrated Masked Autoencoder is applied exclusively to these semantic-rich frames, integrating low-level visual precision with robust high-level semantic representation.
  arXiv  Detail & Related papers  (2026-08-05T05:56:41Z)
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• Dense Feature Learning via Linear Structure Preservation in Medical Data [30.77691570199694]
  Deep learning models for medical data are typically trained using task specific objectives that encourage representations to collapse onto a small number of discriminative directions.<n>We propose dense feature learning, a representation centric framework that explicitly shapes the linear structure of medical embeddings.
  arXiv  Detail & Related papers  (2026-02-07T21:23:35Z)
• PULSE: A Unified Multi-Task Architecture for Cardiac Segmentation, Diagnosis, and Few-Shot Cross-Modality Clinical Adaptation [0.27998963147546135]
  We introduce PULSE, a multi-task vision-language framework built on self-supervised representations and optimized through a composite supervision strategy.<n>A multi-scale token reconstruction decoder enables anatomical segmentation, while shared global representations support disease classification and clinically grounded text output.<n>Unlike prior task-specific pipelines, PULSE learns task-invariant cardiac priors, generalizes robustly across datasets, and can be adapted to new imaging modalities with minimal supervision.
  arXiv  Detail & Related papers  (2025-12-03T14:49:01Z)
• Vision Foundry: A System for Training Foundational Vision AI Models [0.0]
  Vision Foundry is a code-free, HIPAA-compliant platform that democratizes pre-training, adaptation, and deployment of vision models.<n>By bridging the gap between advanced representation learning and practical application, Vision Foundry enables domain experts to develop state-of-the-art clinical AI tools.
  arXiv  Detail & Related papers  (2025-12-03T14:02:22Z)
• Structure is Supervision: Multiview Masked Autoencoders for Radiology [30.875117640663024]
  Building robust medical machine learning systems requires pretraining strategies that exploit the intrinsic structure present in clinical data.<n>We introduce Multiview Masked Autoencoder (MVMAE), a self-supervised framework that learns view-invariant and disease-relevant representations.<n>We extend this approach with MVMAE-V2T, which incorporates radiology reports as an auxiliary text-based learning signal.
  arXiv  Detail & Related papers  (2025-11-27T10:20:51Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• Knowledge to Sight: Reasoning over Visual Attributes via Knowledge Decomposition for Abnormality Grounding [14.354891415427154]
  We propose textbfKnowledge to Sight (K2Sight), a framework that introduces structured supervision by decomposing visual attributes.<n>Unlike conventional report-level supervision, our approach explicitly bridges domain knowledge and spatial structure.<n>We train compact models with 0.23B and 2B semantic parameters using only 1.5% of the data required by state-of-the-art medical VLMs.
  arXiv  Detail & Related papers  (2025-08-06T15:54:44Z)
• Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance [50.486523249499115]
  Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS)<n>We propose Compress-to-Explore (C2E), a novel self-supervised framework to learn compact, informative representations from surgical videos.<n>C2E uses entropy-maximizing decoders to compress images while preserving clinically relevant details, improving encoder performance without labeled data.
  arXiv  Detail & Related papers  (2025-05-16T14:02:24Z)
• Self-adaptive vision-language model for 3D segmentation of pulmonary artery and vein [18.696258519327095]
  This paper proposes a novel framework called Language-guided self-adaptive Cross-Attention Fusion Framework.<n>Our method adopts pre-trained CLIP as a strong feature extractor for generating the segmentation of 3D CT scans.<n>We extensively validate our method on a local dataset, which is the largest pulmonary artery-vein CT dataset to date.
  arXiv  Detail & Related papers  (2025-01-07T12:03:02Z)
• Representation Learning of Structured Data for Medical Foundation Models [29.10129199884847]
  We introduce the UniStruct architecture to design a multimodal medical foundation model of unstructured text and structured data. Our approach is validated through model pre-training on both an extensive internal medical database and a public repository of structured medical records.
  arXiv  Detail & Related papers  (2024-10-17T09:02:28Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning [65.54680361074882]
  Eye-gaze Guided Multi-modal Alignment (EGMA) framework harnesses eye-gaze data for better alignment of medical visual and textual features. We conduct downstream tasks of image classification and image-text retrieval on four medical datasets.
  arXiv  Detail & Related papers  (2024-03-19T03:59:14Z)
• MedContext: Learning Contextual Cues for Efficient Volumetric Medical Segmentation [25.74088298769155]
  We propose a universal training framework called MedContext for 3D medical segmentation. Our approach effectively learns self supervised contextual cues jointly with the supervised voxel segmentation task. The effectiveness of MedContext is validated across multiple 3D medical datasets and four state-of-the-art model architectures.
  arXiv  Detail & Related papers  (2024-02-27T17:58:05Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training [23.506879497561712]
  We employ a contrastive global-local dual-encoder architecture to learn concepts directly from unstructured medical reports. We evaluate our approach on the large-scale chest X-Ray datasets MIMIC-CXR, CheXpert, and ChestX-Ray14 for disease classification.
  arXiv  Detail & Related papers  (2022-05-14T21:44:05Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.