Disentangling Shared and Task-Specific Representations from Multi-Modal Clinical Data

• URL: http://arxiv.org/abs/2605.03570v1
• Date: Tue, 05 May 2026 09:41:36 GMT
• Title: Disentangling Shared and Task-Specific Representations from Multi-Modal Clinical Data
• Abstract summary: We propose a multi-task framework built on a unified Transformer for multimodal fusion to split patient representations into shared and task-specific subspaces.<n>We evaluated OrthTD on a real-world cohort of 12,430 surgical patients for predicting four outcomes.
• Score: 8.493390194430592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world clinical data is inherently multimodal, providing complementary evidence that mirrors the practical necessity of jointly assessing multiple related outcomes. Although multi-task learning can improve efficiency by sharing information across outcomes, existing approaches often fail to balance shared representation learning with outcome-specific modeling. Hard parameter sharing can trigger negative transfer when task gradients conflict, while flexible sharing may still entangle shared and task-specific signals. To address this, we propose a multi-task framework built on a unified Transformer for multimodal fusion, augmented with Orthogonal Task Decomposition (OrthTD) to split patient representations into shared and task-specific subspaces and impose a geometric orthogonality constraint to reduce redundancy and isolate task-specific signals. We evaluated OrthTD on a real-world cohort of 12,430 surgical patients for predicting four outcomes. OrthTD achieved average AUC (area under the receiver operating characteristic curve) of 87.5% and average AUPRC (area under the precision-recall curve) of 37.2%, consistently outperformed advanced tabular and multi-task methods. Notably, OrthTD achieves substantial gains in AUPRC, indicating superior performance in identifying rare events within imbalanced clinical data. These results suggest that enforcing non-redundant shared and task-specific representations can improve multi-outcome prediction from multimodal clinical data.

Related papers

• Capacity and Redundancy Trade-offs in Multi-Task Learning [0.9258421435949676]
  In multi-task learning (MTL) negative transfer is often considered as an optimization artifact, but it can also be viewed as a consequence of weak task redundancy.<n>We investigate this effect through a Capacity--Redundancy (CR) identity that decomposes the sum of per-task predictive informations into joint predictive information.<n>We show two key results: (i) a clustering-gap decomposition that gives a necessary and sufficient condition for clustered sharing to outperform global sharing, and (ii) a gradient--TC bridge in a Gaussian multi-task model that formally justifies gradient cosine similarity as a proxy for
  arXiv  Detail & Related papers  (2026-07-17T23:31:12Z)
• Information-Theoretic Optimization for Task-Adapted Compressed Sensing Magnetic Resonance Imaging [92.90858286761515]
  Task-adapted compressed sensing magnetic resonance imaging (CS-MRI) is emerging to address the specific demands of downstream clinical tasks.<n>We propose the first task-adapted CS-MRI from the information-theoretic perspective to simultaneously achieve probabilistic inference for uncertainty prediction.<n>We leverage amortized optimization and construct tractable variational bounds for mutual information to jointly optimize sampling, reconstruction, and task-inference models.
  arXiv  Detail & Related papers  (2026-04-14T13:23:19Z)
• Understanding Task Aggregation for Generalizable Ultrasound Foundation Models [4.4775919772138515]
  We introduce M2DINO, a multi-organ, multi-task framework built on DINOv3 with task-conditioned Mixture-of-Experts blocks for adaptive capacity allocation.<n>We evaluate 27 ultrasound tasks spanning segmentation, classification, detection, and regression under three paradigms: task-specific, clinically-grouped, and all-task unified training.<n>Our results show that aggregation effectiveness depends strongly on training data scale.
  arXiv  Detail & Related papers  (2026-03-18T16:43:43Z)
• Developing Fairness-Aware Task Decomposition to Improve Equity in Post-Spinal Fusion Complication Prediction [3.860970992977915]
  We propose a fairness-aware multitask learning framework for postoperative complication prediction.<n> FAIR-MTL employs a data-driven subgroup inference mechanism.<n>It achieves an AUC of 0.86 and an accuracy of 75%, outperforming single-task baselines while substantially reducing bias.
  arXiv  Detail & Related papers  (2025-11-29T19:06:07Z)
• MEJO: MLLM-Engaged Surgical Triplet Recognition via Inter- and Intra-Task Joint Optimization [52.149337961205624]
  We propose a framework that empowers both inter- and intra-task optimization for surgical triplet recognition.<n>For inter-task optimization, we introduce the Shared-Specific-Disentangled (S$2$D) learning scheme that decomposes representations into task-shared and task-specific components.<n>For intra-task optimization conflicts, we develop a Coordinated Gradient Learning (CGL) strategy, which dissects and rebalances the positive-negative ambiguities.
  arXiv  Detail & Related papers  (2025-09-16T09:48:52Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• Continually Evolved Multimodal Foundation Models for Cancer Prognosis [50.43145292874533]
  Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates.<n>Previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information.<n>Existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals.<n>Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities.
  arXiv  Detail & Related papers  (2025-01-30T06:49:57Z)
• FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare Prediction [34.732561455987145]
  We propose a unified healthcare prediction model, also named by textbfFlexCare, to flexibly accommodate incomplete multimodal inputs. A task-agnostic multimodal information extraction module is presented to capture decorrelated representations of diverse intra- and inter-modality patterns. Experimental results on multiple tasks from MIMIC-IV/MIMIC-CXR/MIMIC-NOTE datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-06-17T12:03:10Z)
• Multi-Dataset Multi-Task Learning for COVID-19 Prognosis [25.371798627482065]
  We introduce a novel multi-dataset multi-task training framework that predicts COVID-19 prognostic outcomes from chest X-rays. Our framework hypothesizes that assessing severity scores enhances the model's ability to classify prognostic severity groups.
  arXiv  Detail & Related papers  (2024-05-22T15:57:44Z)
• Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning [57.83232242068982]
  Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. It remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. This work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy.
  arXiv  Detail & Related papers  (2023-05-25T15:46:20Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.