The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images

• URL: http://arxiv.org/abs/2211.07254v1
• Date: Mon, 14 Nov 2022 10:32:51 GMT
• Title: The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images
• Authors: Philip M\"uller, Georgios Kaissis, Daniel Rueckert
• Abstract summary: We study how local contrastive losses are related to global (per-sample) contrastive losses and which effects they have on localized medical downstream tasks. Based on a theoretical comparison, we propose to remove some components of local losses and replace others by a novel distribution prior. We empirically study this approach on chest X-ray tasks and find it to be very effective, outperforming methods without local losses on 12 of 18 tasks.
• Score: 7.49320945341034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-text contrastive learning has proven effective for pretraining medical image models. When targeting localized downstream tasks like semantic segmentation or object detection, additional local contrastive losses that align image regions with sentences have shown promising results. We study how local contrastive losses are related to global (per-sample) contrastive losses and which effects they have on localized medical downstream tasks. Based on a theoretical comparison, we propose to remove some components of local losses and replace others by a novel distribution prior which enforces uniformity of representations within each sample. We empirically study this approach on chest X-ray tasks and find it to be very effective, outperforming methods without local losses on 12 of 18 tasks.

Related papers

• Visual Context-Aware Person Fall Detection [52.49277799455569]
  We present a segmentation pipeline to semi-automatically separate individuals and objects in images. Background objects such as beds, chairs, or wheelchairs can challenge fall detection systems, leading to false positive alarms. We demonstrate that object-specific contextual transformations during training effectively mitigate this challenge.
  arXiv  Detail & Related papers  (2024-04-11T19:06:36Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Annotation by Clicks: A Point-Supervised Contrastive Variance Method for Medical Semantic Segmentation [38.61378161105941]
  We propose a novel point-supervised contrastive variance method (PSCV) for medical image semantic segmentation. PSCV only requires one pixel-point from each organ category to be annotated. We show that the proposed method outperforms the state-of-the-art weakly supervised methods on point-supervised medical image semantic segmentation tasks.
  arXiv  Detail & Related papers  (2022-12-17T01:07:21Z)
• What can we learn about a generated image corrupting its latent representation? [57.1841740328509]
  We investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck. We achieve this by corrupting the latent representation with noise and generating multiple outputs.
  arXiv  Detail & Related papers  (2022-10-12T14:40:32Z)
• PCA: Semi-supervised Segmentation with Patch Confidence Adversarial Training [52.895952593202054]
  We propose a new semi-supervised adversarial method called Patch Confidence Adrial Training (PCA) for medical image segmentation. PCA learns the pixel structure and context information in each patch to get enough gradient feedback, which aids the discriminator in convergent to an optimal state. Our method outperforms the state-of-the-art semi-supervised methods, which demonstrates its effectiveness for medical image segmentation.
  arXiv  Detail & Related papers  (2022-07-24T07:45:47Z)
• Leveraging Uncertainty for Deep Interpretable Classification and Weakly-Supervised Segmentation of Histology Images [25.429124017422385]
  Deep weakly supervised methods allow image classification and ROI segmentation for interpretability. These methods lack mechanisms for modeling explicitly non-discriminative regions which raises false-positive rates. We propose novel regularization terms, which enable the model to seek both non-discriminative and discriminative regions.
  arXiv  Detail & Related papers  (2022-05-12T02:18:23Z)
• Robust Medical Image Classification from Noisy Labeled Data with Global and Local Representation Guided Co-training [73.60883490436956]
  We propose a novel collaborative training paradigm with global and local representation learning for robust medical image classification. We employ the self-ensemble model with a noisy label filter to efficiently select the clean and noisy samples. We also design a novel global and local representation learning scheme to implicitly regularize the networks to utilize noisy samples.
  arXiv  Detail & Related papers  (2022-05-10T07:50:08Z)
• Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation [13.996217500923413]
  Semi/self-supervised learning-based approaches exploit unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images. We propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information.
  arXiv  Detail & Related papers  (2021-12-17T17:38:56Z)
• Joint Learning of Localized Representations from Medical Images and Reports [0.0]
  We propose Localized representation learning from Vision and Text (LoVT) to target localized medical imaging tasks. Our method combines instance-level image-report contrastive learning with local contrastive learning on image region and report sentence representations. LoVT performs best on 11 out of the 18 studied tasks making it the preferred method of choice for localized tasks.
  arXiv  Detail & Related papers  (2021-12-06T09:27:24Z)
• Unsupervised Local Discrimination for Medical Images [35.14445357879895]
  Contrastive representation learning is an effective method to alleviate the demand for expensive annotated data in medical image processing. Recent work mainly based on instance-wise discrimination to learn global features, while neglect local details. We propose a universal local discrmination framework to learn local discriminative features to effectively initialize medical models.
  arXiv  Detail & Related papers  (2021-08-21T04:53:19Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.