ADR: Attention Diversification Regularization for Mitigating Overfitting in Multiple Instance Learning based Whole Slide Image Classification

• URL: http://arxiv.org/abs/2406.15303v1
• Date: Tue, 18 Jun 2024 02:01:17 GMT
• Title: ADR: Attention Diversification Regularization for Mitigating Overfitting in Multiple Instance Learning based Whole Slide Image Classification
• Authors: Yunlong Zhang, Zhongyi Shui, Yunxuan Sun, Honglin Li, Jingxiong Li, Chenglu Zhu, Sunyi Zheng, Lin Yang,
• Abstract summary: Multiple Instance Learning (MIL) has demonstrated effectiveness in analyzing whole slide images (WSIs) This paper reveals the correlation between MIL's performance and the entropy of attention values. We propose Attention Diversity Regularization (ADR), a technique aimed at promoting high entropy in attention values.
• Score: 8.746388561544205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multiple Instance Learning (MIL) has demonstrated effectiveness in analyzing whole slide images (WSIs), yet it often encounters overfitting challenges in real-world applications. This paper reveals the correlation between MIL's performance and the entropy of attention values. Based on this observation, we propose Attention Diversity Regularization (ADR), a simple but effective technique aimed at promoting high entropy in attention values. Specifically, ADR introduces a negative Shannon entropy loss for attention values into the regular MIL framework. Compared to existing methods aimed at alleviating overfitting, which often necessitate additional modules or processing steps, our ADR approach requires no such extras, demonstrating simplicity and efficiency. We evaluate our ADR on three WSI classification tasks. ADR achieves superior performance over the state-of-the-art on most of them. We also show that ADR can enhance heatmaps, aligning them better with pathologists' diagnostic criteria. The source code is available at \url{https://github.com/dazhangyu123/ADR}.

Related papers

• CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Prompting [48.47935559597376]
  Diabetic retinopathy (DR) is a complication of diabetes and usually takes decades to reach sight-threatening levels. Most current DR grading methods suffer from insufficient robustness to data variability. We propose a novel DR grading framework CLIP-DR based on three observations.
  arXiv  Detail & Related papers  (2024-07-04T17:14:18Z)
• ADR-BC: Adversarial Density Weighted Regression Behavior Cloning [29.095342729527733]
  Imitation Learning (IL) methods first shape a reward or Q function and then use this shaped function within a reinforcement learning framework to optimize the empirical policy. We propose ADR-BC, which aims to enhance behavior cloning through augmented density-based action support. As a one-step behavior cloning framework, ADR-BC avoids the cumulative bias associated with multi-step RL frameworks.
  arXiv  Detail & Related papers  (2024-05-28T06:59:16Z)
• Inverse-RLignment: Inverse Reinforcement Learning from Demonstrations for LLM Alignment [62.05713042908654]
  We introduce Alignment from Demonstrations (AfD), a novel approach leveraging high-quality demonstration data to overcome these challenges. We formalize AfD within a sequential decision-making framework, highlighting its unique challenge of missing reward signals. Practically, we propose a computationally efficient algorithm that extrapolates over a tailored reward model for AfD.
  arXiv  Detail & Related papers  (2024-05-24T15:13:53Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• ARHNet: Adaptive Region Harmonization for Lesion-aware Augmentation to Improve Segmentation Performance [61.04246102067351]
  We propose a foreground harmonization framework (ARHNet) to tackle intensity disparities and make synthetic images look more realistic. We demonstrate the efficacy of our method in improving the segmentation performance using real and synthetic images.
  arXiv  Detail & Related papers  (2023-07-02T10:39:29Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• ASR: Attention-alike Structural Re-parameterization [56.88192537044364]
  We propose a simple-yet-effective attention-alike structural re- parameterization (ASR) that allows us to achieve SRP for a given network while enjoying the effectiveness of the attention mechanism. In this paper, we conduct extensive experiments from a statistical perspective and discover an interesting phenomenon Stripe Observation, which reveals that channel attention values quickly approach some constant vectors during training.
  arXiv  Detail & Related papers  (2023-04-13T08:52:34Z)
• A Global and Patch-wise Contrastive Loss for Accurate Automated Exudate Detection [12.669734891001667]
  Diabetic retinopathy (DR) is a leading global cause of blindness. Early detection of hard exudates plays a crucial role in identifying DR, which aids in treating diabetes and preventing vision loss. We present a novel supervised contrastive learning framework to optimize hard exudate segmentation.
  arXiv  Detail & Related papers  (2023-02-22T17:39:00Z)
• Self-Supervised Equivariant Regularization Reconciles Multiple Instance Learning: Joint Referable Diabetic Retinopathy Classification and Lesion Segmentation [3.1671604920729224]
  Lesion appearance is a crucial clue for medical providers to distinguish referable diabetic retinopathy (rDR) from non-referable DR. Most existing large-scale DR datasets contain only image-level labels rather than pixel-based annotations. This paper leverages self-supervised equivariant learning and attention-based multi-instance learning to tackle this problem. We conduct extensive validation experiments on the Eyepacs dataset, achieving an area under the receiver operating characteristic curve (AU ROC) of 0.958, outperforming current state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2022-10-12T06:26:05Z)
• Multi-level Distance Regularization for Deep Metric Learning [20.178765779788492]
  We propose a novel distance-based regularization method for deep metric learning called Multi-level Distance Regularization (MDR) MDR explicitly disturbs a learning procedure by regularizing pairwise distances between embedding vectors into multiple levels. By easily adopting our MDR, the previous approaches can be improved in performance and generalization ability.
  arXiv  Detail & Related papers  (2021-02-08T14:16:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.