Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings

• URL: http://arxiv.org/abs/2104.10715v1
• Date: Wed, 21 Apr 2021 18:28:13 GMT
• Title: Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings
• Authors: Utkarsh Sarawgi, Rishab Khincha, Wazeer Zulfikar, Satrajit Ghosh, Pattie Maes
• Abstract summary: Uncertainty estimation is a widely researched method to highlight the confidence of machine learning systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings. We propose an uncertainty-aware boosting technique for multi-modal ensembling in order to focus on the data points with higher associated uncertainty estimates.
• Score: 33.25969141014772
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliability of machine learning (ML) systems is crucial in safety-critical applications such as healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of ML systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings by leveraging the feature sets together. We propose an uncertainty-aware boosting technique for multi-modal ensembling in order to focus on the data points with higher associated uncertainty estimates, rather than the ones with higher loss values. We evaluate this method on healthcare tasks related to Dementia and Parkinson's disease which involve real-world multi-modal speech and text data, wherein our method shows an improved performance. Additional analysis suggests that introducing uncertainty-awareness into the boosted ensembles decreases the overall entropy of the system, making it more robust to heteroscedasticity in the data, as well as better calibrating each of the modalities along with high quality prediction intervals. We open-source our entire codebase at https://github.com/usarawgi911/Uncertainty-aware-boosting

Related papers

• Confidence-aware multi-modality learning for eye disease screening [58.861421804458395]
  We propose a novel multi-modality evidential fusion pipeline for eye disease screening. It provides a measure of confidence for each modality and elegantly integrates the multi-modality information. Experimental results on both public and internal datasets demonstrate that our model excels in robustness.
  arXiv  Detail & Related papers  (2024-05-28T13:27:30Z)
• Enhancing Person Re-Identification via Uncertainty Feature Fusion and Auto-weighted Measure Combination [1.2923961938782627]
  This study presents a novel methodology that significantly enhances Person Re-Identification (Re-ID) Tested on benchmark datasets - Market-1501, DukeMTMC-ReID, and MSMT17 - our approach demonstrates substantial improvements in Rank-1 accuracy and mean Average Precision (mAP) UFFM capitalizes on the power of feature synthesis from multiple images to overcome the limitations imposed by the variability of subject appearances across different views. WDA further refines the process by intelligently aggregating similarity metrics, thereby enhancing the system's ability to discern subtle but critical differences between subjects.
  arXiv  Detail & Related papers  (2024-05-02T09:09:48Z)
• Machine Learning Robustness: A Primer [12.426425119438846]
  The discussion begins with a detailed definition of robustness, portraying it as the ability of ML models to maintain stable performance across varied and unexpected environmental conditions. The chapter delves into the factors that impede robustness, such as data bias, model complexity, and the pitfalls of underspecified ML pipelines. The discussion progresses to explore amelioration strategies for bolstering robustness, starting with data-centric approaches like debiasing and augmentation.
  arXiv  Detail & Related papers  (2024-04-01T03:49:42Z)
• A Holistic Assessment of the Reliability of Machine Learning Systems [30.638615396429536]
  This paper proposes a holistic assessment methodology for the reliability of machine learning (ML) systems. Our framework evaluates five key properties: in-distribution accuracy, distribution-shift robustness, adversarial robustness, calibration, and out-of-distribution detection. To provide insights into the performance of different algorithmic approaches, we identify and categorize state-of-the-art techniques.
  arXiv  Detail & Related papers  (2023-07-20T05:00:13Z)
• Unsupervised Quality Prediction for Improved Single-Frame and Weighted Sequential Visual Place Recognition [20.737660223671003]
  We present a new, training-free approach to predicting the likely quality of localization estimates. We use these predictions to bias a sequence-matching process to produce additional performance gains. Our system is lightweight, runs in real-time and is agnostic to the underlying VPR technique.
  arXiv  Detail & Related papers  (2023-07-04T03:53:05Z)
• Calibrating Multimodal Learning [94.65232214643436]
  We propose a novel regularization technique, i.e., Calibrating Multimodal Learning (CML) regularization, to calibrate the predictive confidence of previous methods. This technique could be flexibly equipped by existing models and improve the performance in terms of confidence calibration, classification accuracy, and model robustness.
  arXiv  Detail & Related papers  (2023-06-02T04:29:57Z)
• Quality-Based Conditional Processing in Multi-Biometrics: Application to Sensor Interoperability [63.05238390013457]
  We describe and evaluate the ATVS-UAM fusion approach submitted to the quality-based evaluation of the 2007 BioSecure Multimodal Evaluation Campaign. Our approach is based on linear logistic regression, in which fused scores tend to be log-likelihood-ratios. Results show that the proposed approach outperforms all the rule-based fusion schemes.
  arXiv  Detail & Related papers  (2022-11-24T12:11:22Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Uncertainty-Aware Multi-Modal Ensembling for Severity Prediction of Alzheimer's Dementia [39.29536042476913]
  We propose an uncertainty-aware boosting technique for multi-modal ensembling to predict Alzheimer's Dementia Severity. The propagation of uncertainty across acoustic, cognitive, and linguistic features produces an ensemble system robust to heteroscedasticity in the data.
  arXiv  Detail & Related papers  (2020-10-03T23:08:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.