PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers

• URL: http://arxiv.org/abs/2308.10112v2
• Date: Fri, 24 May 2024 03:46:08 GMT
• Title: PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers
• Authors: Wenhui Zhu, Peijie Qiu, Xiwen Chen, Oana M. Dumitrascu, Yalin Wang,
• Abstract summary: Multiple instance learning (MIL) was a weakly supervised learning approach that sought to assign binary class labels to collections of instances known as bags. We present a novel approach in the form of a Progressive Dropout Layer (PDL) to address overfitting and empower the MIL model in uncovering intricate and impactful feature representations.
• Score: 2.069061136213899
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multiple instance learning (MIL) was a weakly supervised learning approach that sought to assign binary class labels to collections of instances known as bags. However, due to their weak supervision nature, the MIL methods were susceptible to overfitting and required assistance in developing comprehensive representations of target instances. While regularization typically effectively combated overfitting, its integration with the MIL model has been frequently overlooked in prior studies. Meanwhile, current regularization methods for MIL have shown limitations in their capacity to uncover a diverse array of representations. In this study, we delve into the realm of regularization within the MIL model, presenting a novel approach in the form of a Progressive Dropout Layer (PDL). We aim to not only address overfitting but also empower the MIL model in uncovering intricate and impactful feature representations. The proposed method was orthogonal to existing MIL methods and could be easily integrated into them to boost performance. Our extensive evaluation across a range of MIL benchmark datasets demonstrated that the incorporation of the PDL into multiple MIL methods not only elevated their classification performance but also augmented their potential for weakly-supervised feature localizations.

Related papers

• Attention Is Not What You Need: Revisiting Multi-Instance Learning for Whole Slide Image Classification [51.95824566163554]
  We argue that synergizing the standard MIL assumption with variational inference encourages the model to focus on tumour morphology instead of spurious correlations. Our method also achieves better classification boundaries for identifying hard instances and mitigates the effect of spurious correlations between bags and labels.
  arXiv  Detail & Related papers  (2024-08-18T12:15:22Z)
• Hyperbolic Secant representation of the logistic function: Application to probabilistic Multiple Instance Learning for CT intracranial hemorrhage detection [0.0]
  Multiple Instance Learning (MIL) is a weakly supervised paradigm that has been successfully applied to many different scientific areas. We propose a general GP-based MIL method that takes different forms by simply leveraging distributions other than the Hyperbolic Secant one. This is validated in a comprehensive experimental study including one synthetic MIL dataset, two well-known MIL benchmarks, and a real-world medical problem.
  arXiv  Detail & Related papers  (2024-03-21T20:43:34Z)
• MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology [10.933433327636918]
  Multiple Instance Learning (MIL) has emerged as a dominant paradigm to extract discriminative feature representations within Whole Slide Images (WSIs) in computational pathology. In this paper, we incorporate the Selective Scan Space State Sequential Model (Mamba) in Multiple Instance Learning (MIL) for long sequence modeling with linear complexity. Our proposed framework performs favorably against state-of-the-art MIL methods.
  arXiv  Detail & Related papers  (2024-03-11T15:17:25Z)
• Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests [59.623267208433255]
  Multiple Instance Learning (MIL) is a sub-domain of classification problems with positive and negative labels and a "bag" of inputs. In this work, we examine five of the most prominent deep-MIL models and find that none of them respects the standard MIL assumption. We identify and demonstrate this problem via a proposed "algorithmic unit test", where we create synthetic datasets that can be solved by a MIL respecting model.
  arXiv  Detail & Related papers  (2023-10-27T03:05:11Z)
• Combating Representation Learning Disparity with Geometric Harmonization [50.29859682439571]
  We propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning. Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner.
  arXiv  Detail & Related papers  (2023-10-26T17:41:11Z)
• Multiple Instance Learning Framework with Masked Hard Instance Mining for Whole Slide Image Classification [11.996318969699296]
  Masked hard instance mining (MHIM-MIL) is presented. MHIM-MIL uses a Siamese structure (Teacher-Student) with a consistency constraint to explore potential hard instances. Experimental results on the CAMELYON-16 and TCGA Lung Cancer datasets demonstrate that MHIM-MIL outperforms other latest methods in terms of performance and training cost.
  arXiv  Detail & Related papers  (2023-07-28T01:40:04Z)
• Feature Re-calibration based MIL for Whole Slide Image Classification [7.92885032436243]
  Whole slide image (WSI) classification is a fundamental task for the diagnosis and treatment of diseases. We propose to re-calibrate the distribution of a WSI bag (instances) by using the statistics of the max-instance (critical) feature. We employ a position encoding module (PEM) to model spatial/morphological information, and perform pooling by multi-head self-attention (PSMA) with a Transformer encoder.
  arXiv  Detail & Related papers  (2022-06-22T07:00:39Z)
• ProtoMIL: Multiple Instance Learning with Prototypical Parts for Fine-Grained Interpretability [2.094672430475796]
  Multiple Instance Learning (MIL) gains popularity in many real-life machine learning applications due to its weakly supervised nature. In this paper, we introduce ProtoMIL, a novel self-explainable MIL method inspired by the case-based reasoning process that operates on visual prototypes. Thanks to incorporating prototypical features into objects description, ProtoMIL unprecedentedly joins the model accuracy and fine-grained interpretability.
  arXiv  Detail & Related papers  (2021-08-24T10:02:31Z)
• CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction [52.94486705393062]
  We go beyond typical multi-instance learning (MIL) framework and propose a novel contrastive instance learning (CIL) framework. Specifically, we regard the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previous methods on NYT10, GDS and KBP.
  arXiv  Detail & Related papers  (2021-06-21T04:51:59Z)
• Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training [53.347081723070666]
  We propose collaborative adversarial training to improve the data utilization of multi-instance learning (MIL) Since VAT is label-free, we employ the instance-level VAT to recycle instances abandoned by MIL. Our proposed method brings consistent improvements ( 5 absolute AUC score) to the previous state of the art.
  arXiv  Detail & Related papers  (2021-06-21T03:54:02Z)
• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning [63.64636047748605]
  We develop a new theoretical framework to provide convergence guarantee for the general multi-step MAML algorithm. In particular, our results suggest that an inner-stage step needs to be chosen inversely proportional to $N$ of inner-stage steps in order for $N$ MAML to have guaranteed convergence.
  arXiv  Detail & Related papers  (2020-02-18T19:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.