Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective

• URL: http://arxiv.org/abs/2604.05651v1
• Date: Tue, 07 Apr 2026 09:54:37 GMT
• Title: Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective
• Abstract summary: We investigate how medical vision tasks relate, overlap, or differ on a representational level.<n>Our goal is to probe whether a data-driven representation space can capture an underlying structure of tasks.<n>We introduce Task-Contrastive Learning (TaCo), a contrastive learning framework designed to embed tasks into a shared representation space.
• Score: 7.992671861007712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While much of the medical computer vision community has focused on advancing performance for specific tasks, the underlying relationships between tasks, i.e., how they relate, overlap, or differ on a representational level, remain largely unexplored. Our work explores these intrinsic relationships between medical vision tasks, specifically, we investigate 30 tasks, such as semantic tasks (e.g., segmentation and detection), image generative tasks (e.g., denoising, inpainting, or colorization), and image transformation tasks (e.g., geometric transformations). Our goal is to probe whether a data-driven representation space can capture an underlying structure of tasks across a variety of 39 datasets from wildly different medical imaging modalities, including computed tomography, magnetic resonance, electron microscopy, X-ray ultrasound and more. By revealing how tasks relate to one another, we aim to provide insights into their fundamental properties and interconnectedness. To this end, we introduce Task-Contrastive Learning (TaCo), a contrastive learning framework designed to embed tasks into a shared representation space. Through TaCo, we map these heterogeneous tasks from different modalities into a joint space and analyze their properties: identifying which tasks are distinctly represented, which blend together, and how iterative alterations to tasks are reflected in the embedding space. Our work provides a foundation for understanding the intrinsic structure of medical vision tasks, offering a deeper understanding of task similarities and their interconnected properties in embedding spaces.

Related papers

• Is Visual in-Context Learning for Compositional Medical Tasks within Reach? [68.56630652862293]
  In this paper, we explore the potential of visual in-context learning to enable a single model to handle multiple tasks.<n>We introduce a novel method for training in-context learners using a synthetic compositional task generation engine.
  arXiv  Detail & Related papers  (2025-07-01T15:32:23Z)
• Reverse Probing: Evaluating Knowledge Transfer via Finetuned Task Embeddings for Coreference Resolution [23.375053899418504]
  Instead of probing frozen representations from a complex source task, we explore the effectiveness of embeddings from multiple simple source tasks on a single target task.<n>Our findings reveal that task embeddings vary significantly in utility for coreference resolution, with semantic similarity tasks proving most beneficial.
  arXiv  Detail & Related papers  (2025-01-31T17:12:53Z)
• YOLO-MED : Multi-Task Interaction Network for Biomedical Images [18.535117490442953]
  YOLO-Med is an efficient end-to-end multi-task network capable of concurrently performing object detection and semantic segmentation. Our model exhibits promising results in balancing accuracy and speed when evaluated on the Kvasir-seg dataset and a private biomedical image dataset.
  arXiv  Detail & Related papers  (2024-03-01T03:20:42Z)
• Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond [62.406687088097605]
  Multi-Task Learning (MTL) is a framework, where multiple related tasks are learned jointly and benefit from a shared representation space. We show that MTL can be successful with classification tasks with little, or non-overlapping annotations. We propose a novel approach, where knowledge exchange is enabled between the tasks via distribution matching.
  arXiv  Detail & Related papers  (2024-01-02T14:18:11Z)
• InstructDiffusion: A Generalist Modeling Interface for Vision Tasks [52.981128371910266]
  We present InstructDiffusion, a framework for aligning computer vision tasks with human instructions. InstructDiffusion could handle a variety of vision tasks, including understanding tasks and generative tasks. It even exhibits the ability to handle unseen tasks and outperforms prior methods on novel datasets.
  arXiv  Detail & Related papers  (2023-09-07T17:56:57Z)
• A Dynamic Feature Interaction Framework for Multi-task Visual Perception [100.98434079696268]
  We devise an efficient unified framework to solve multiple common perception tasks. These tasks include instance segmentation, semantic segmentation, monocular 3D detection, and depth estimation. Our proposed framework, termed D2BNet, demonstrates a unique approach to parameter-efficient predictions for multi-task perception.
  arXiv  Detail & Related papers  (2023-06-08T09:24:46Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• UniNet: A Unified Scene Understanding Network and Exploring Multi-Task Relationships through the Lens of Adversarial Attacks [1.1470070927586016]
  Single task vision networks extract information only based on some aspects of the scene. In multi-task learning (MTL), single tasks are jointly learned, thereby providing an opportunity for tasks to share information. We develop UniNet, a unified scene understanding network that accurately and efficiently infers vital vision tasks.
  arXiv  Detail & Related papers  (2021-08-10T11:00:56Z)
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study [75.42182503265056]
  Multi-Task Learning has emerged as a methodology in which multiple tasks are jointly learned by a shared learning algorithm. We deal with heterogeneous MTL, simultaneously addressing detection, classification & regression problems. We build FaceBehaviorNet, the first framework for large-scale face analysis, by jointly learning all facial behavior tasks.
  arXiv  Detail & Related papers  (2021-05-08T22:26:52Z)
• Taskology: Utilizing Task Relations at Scale [28.09712466727001]
  We show that we can leverage the inherent relationships among collections of tasks, as they are trained jointly. explicitly utilizing the relationships between tasks allows improving their performance while dramatically reducing the need for labeled data. We demonstrate our framework on subsets of the following collection of tasks: depth and normal prediction, semantic segmentation, 3D motion and ego-motion estimation, and object tracking and 3D detection in point clouds.
  arXiv  Detail & Related papers  (2020-05-14T22:53:46Z)
• Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton [108.01007935498104]
  In this paper, we solve three low-level pixel-wise vision problems, including salient object segmentation, edge detection, and skeleton extraction. We first show some similarities shared by these tasks and then demonstrate how they can be leveraged for developing a unified framework.
  arXiv  Detail & Related papers  (2020-04-18T11:10:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.