CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation

• URL: http://arxiv.org/abs/2410.19390v1
• Date: Fri, 25 Oct 2024 08:46:55 GMT
• Title: CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation
• Authors: Qiufan Lin, Hengxin Ruan, Dominique Fouchez, Shupei Chen, Rui Li, Paulo Montero-Camacho, Nicola R. Napolitano, Yuan-Sen Ting, Wei Zhang,
• Abstract summary: We develop a novel method called the Contrastive Learning and Adaptive KNN for Photometric Redshift (CLAP) It leverages supervised contrastive learning (SCL) and k-nearest neighbours (KNN) to construct and calibrate raw probability density estimates. The harmonic mean is adopted to combine an ensemble of estimates from multiple realisations for improving accuracy.
• Score: 3.611102630303458
• License:
• Abstract: Obtaining well-calibrated photometric redshift probability densities for galaxies without a spectroscopic measurement remains a challenge. Deep learning discriminative models, typically fed with multi-band galaxy images, can produce outputs that mimic probability densities and achieve state-of-the-art accuracy. However, such models may be affected by miscalibration that would result in discrepancies between the model outputs and the actual distributions of true redshifts. Our work develops a novel method called the Contrastive Learning and Adaptive KNN for Photometric Redshift (CLAP) that resolves this issue. It leverages supervised contrastive learning (SCL) and k-nearest neighbours (KNN) to construct and calibrate raw probability density estimates, and implements a refitting procedure to resume end-to-end discriminative models ready to produce final estimates for large-scale imaging data. The harmonic mean is adopted to combine an ensemble of estimates from multiple realisations for improving accuracy. Our experiments demonstrate that CLAP takes advantage of both deep learning and KNN, outperforming benchmark methods on the calibration of probability density estimates and retaining high accuracy and computational efficiency. With reference to CLAP, we point out that miscalibration is particularly sensitive to the method-induced excessive correlations among data instances in addition to the unaccounted-for epistemic uncertainties. Reducing the uncertainties may not guarantee the removal of miscalibration due to the presence of such excessive correlations, yet this is a problem for conventional deep learning methods rather than CLAP. These discussions underscore the robustness of CLAP for obtaining photometric redshift probability densities required by astrophysical and cosmological applications. This is the first paper in our series on CLAP.

Related papers

• Calibrating Deep Neural Network using Euclidean Distance [5.675312975435121]
  In machine learning, Focal Loss is commonly used to reduce misclassification rates by emphasizing hard-to-classify samples. High calibration error indicates a misalignment between predicted probabilities and actual outcomes, affecting model reliability. This research introduces a novel loss function called Focal Loss (FCL), designed to improve probability calibration while retaining the advantages of Focal Loss in handling difficult samples.
  arXiv  Detail & Related papers  (2024-10-23T23:06:50Z)
• Revisiting Essential and Nonessential Settings of Evidential Deep Learning [70.82728812001807]
  Evidential Deep Learning (EDL) is an emerging method for uncertainty estimation. We propose Re-EDL, a simplified yet more effective variant of EDL.
  arXiv  Detail & Related papers  (2024-10-01T04:27:07Z)
• Achieving Well-Informed Decision-Making in Drug Discovery: A Comprehensive Calibration Study using Neural Network-Based Structure-Activity Models [4.619907534483781]
  computational models that predict drug-target interactions are valuable tools to accelerate the development of new therapeutic agents. However, such models can be poorly calibrated, which results in unreliable uncertainty estimates. We show that combining post hoc calibration method with well-performing uncertainty quantification approaches can boost model accuracy and calibration.
  arXiv  Detail & Related papers  (2024-07-19T10:29:00Z)
• Qubit Dynamics beyond Lindblad: Non-Markovianity versus Rotating Wave Approximation [0.0]
  Even subtle effects in the interaction between qubits and environmental degrees of freedom become progressively relevant and experimentally visible. This applies particularly to the timescale separations that are at the basis of the most commonly used numerical simulation platform for qubit operations. We shed light on the questions (i) to which extent it is possible to monitor violations of either of these timescale separations experimentally and (ii) which of them is the most severe to provide highly accurate predictions.
  arXiv  Detail & Related papers  (2023-08-11T09:16:07Z)
• Parametric and Multivariate Uncertainty Calibration for Regression and Object Detection [4.630093015127541]
  We show that common detection models overestimate the spatial uncertainty in comparison to the observed error. Our experiments show that the simple Isotonic Regression recalibration method is sufficient to achieve a good calibrated uncertainty. In contrast, if normal distributions are required for subsequent processes, our GP-Normal recalibration method yields the best results.
  arXiv  Detail & Related papers  (2022-07-04T08:00:20Z)
• Photometric Redshift Estimation with Convolutional Neural Networks and Galaxy Images: A Case Study of Resolving Biases in Data-Driven Methods [0.0]
  We investigate two major forms of biases, i.e., class-dependent residuals and mode collapse, in a case study of estimating photometric redshifts. We propose a set of consecutive steps for resolving the two biases based on CNN models. Experiments show that our methods possess a better capability in controlling biases compared to benchmark methods.
  arXiv  Detail & Related papers  (2022-02-21T02:59:33Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Differentiable Annealed Importance Sampling and the Perils of Gradient Noise [68.44523807580438]
  Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation. Differentiability is a desirable property as it would admit the possibility of optimizing marginal likelihood as an objective. We propose a differentiable algorithm by abandoning Metropolis-Hastings steps, which further unlocks mini-batch computation.
  arXiv  Detail & Related papers  (2021-07-21T17:10:14Z)
• Transferable Calibration with Lower Bias and Variance in Domain Adaptation [139.4332115349543]
  Domain Adaptation (DA) enables transferring a learning machine from a labeled source domain to an unlabeled target one. How to estimate the predictive uncertainty of DA models is vital for decision-making in safety-critical scenarios. TransCal can be easily applied to recalibrate existing DA methods.
  arXiv  Detail & Related papers  (2020-07-16T11:09:36Z)
• Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels [92.98756432746482]
  We study a weakly supervised problem called learning with complementary labels. We show that the quality of gradient estimation matters more in risk minimization. We propose a novel surrogate complementary loss(SCL) framework that trades zero bias with reduced variance.
  arXiv  Detail & Related papers  (2020-07-05T04:19:37Z)
• Calibration of Neural Networks using Splines [51.42640515410253]
  Measuring calibration error amounts to comparing two empirical distributions. We introduce a binning-free calibration measure inspired by the classical Kolmogorov-Smirnov (KS) statistical test. Our method consistently outperforms existing methods on KS error as well as other commonly used calibration measures.
  arXiv  Detail & Related papers  (2020-06-23T07:18:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.