Using different sources of ground truths and transfer learning to improve the generalization of photometric redshift estimation

• URL: http://arxiv.org/abs/2411.18054v1
• Date: Wed, 27 Nov 2024 04:55:37 GMT
• Title: Using different sources of ground truths and transfer learning to improve the generalization of photometric redshift estimation
• Authors: Jonathan Soriano, Srinath Saikrishnan, Vikram Seenivasan, Bernie Boscoe, Jack Singal, Tuan Do,
• Abstract summary: We explore methods to improve galaxy redshift predictions by combining different ground truths. We first train a base neural network on TransferZ and then refine it using transfer learning on a dataset of galaxies with more precise spectroscopic redshifts (GalaxiesML) Both methods reduce bias by $sim$ 5x, RMS error by $sim$ 1.5x, and catastrophic outlier rates by 1.3x on GalaxiesML, compared to a baseline trained only on TransferZ data.
• Score: 0.0
• License:
• Abstract: In this work, we explore methods to improve galaxy redshift predictions by combining different ground truths. Traditional machine learning models rely on training sets with known spectroscopic redshifts, which are precise but only represent a limited sample of galaxies. To make redshift models more generalizable to the broader galaxy population, we investigate transfer learning and directly combining ground truth redshifts derived from photometry and spectroscopy. We use the COSMOS2020 survey to create a dataset, TransferZ, which includes photometric redshift estimates derived from up to 35 imaging filters using template fitting. This dataset spans a wider range of galaxy types and colors compared to spectroscopic samples, though its redshift estimates are less accurate. We first train a base neural network on TransferZ and then refine it using transfer learning on a dataset of galaxies with more precise spectroscopic redshifts (GalaxiesML). In addition, we train a neural network on a combined dataset of TransferZ and GalaxiesML. Both methods reduce bias by $\sim$ 5x, RMS error by $\sim$ 1.5x, and catastrophic outlier rates by 1.3x on GalaxiesML, compared to a baseline trained only on TransferZ. However, we also find a reduction in performance for RMS and bias when evaluated on TransferZ data. Overall, our results demonstrate these approaches can meet cosmological requirements.

Related papers

• Mantis Shrimp: Exploring Photometric Band Utilization in Computer Vision Networks for Photometric Redshift Estimation [0.30924355683504173]
  We present a model for photometric redshift estimation that fuses ultra-violet (GALEX), optical (PanSTARRS), and infrared (UnWISE) imagery. Mantis Shrimp estimates the conditional density estimate of redshift using cutout images. We study how the models learn to use information across bands, finding evidence that our models successfully incorporates information from all surveys.
  arXiv  Detail & Related papers  (2025-01-15T19:46:23Z)
• Determination of galaxy photometric redshifts using Conditional Generative Adversarial Networks (CGANs) [0.0]
  We present a new algorithmic approach for determining photometric redshifts of galaxies using Conditional Generative Adversarial Networks (CGANs) Proposed CGAN implementation, approaches photometric redshift determination as a probabilistic regression, where instead of determining a single value for the estimated redshift of the galaxy, a full probability density is computed.
  arXiv  Detail & Related papers  (2025-01-11T12:42:07Z)
• GalaxiesML: a dataset of galaxy images, photometry, redshifts, and structural parameters for machine learning [1.0279580671257864]
  We present a dataset built for machine learning applications consisting of galaxy photometry, images, spectroscopic redshifts, and structural properties. This dataset comprises 286,401 galaxy images and photometry from the Hyper-Suprime-Cam Survey PDR2 in five imaging filters. We make this dataset public to help spur development of machine learning methods for the next generation of surveys such as Euclid and LSST.
  arXiv  Detail & Related papers  (2024-09-30T22:46:44Z)
• Deep Learning Based Speckle Filtering for Polarimetric SAR Images. Application to Sentinel-1 [51.404644401997736]
  We propose a complete framework to remove speckle in polarimetric SAR images using a convolutional neural network. Experiments show that the proposed approach offers exceptional results in both speckle reduction and resolution preservation.
  arXiv  Detail & Related papers  (2024-08-28T10:07:17Z)
• SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection [53.19618419772467]
  Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds. With the development of Transformer, the scale of SIRST models is constantly increasing. With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
  arXiv  Detail & Related papers  (2024-03-08T16:14:54Z)
• Photo-zSNthesis: Converting Type Ia Supernova Lightcurves to Redshift Estimates via Deep Learning [0.0]
  Photo-zSNthesis is a convolutional neural network-based method for predicting full redshift probability distributions. We show a 61x improvement in prediction bias Delta z> on PLAsTiCC simulations and 5x improvement on real SDSS data.
  arXiv  Detail & Related papers  (2023-05-19T17:59:00Z)
• Cosmology from Galaxy Redshift Surveys with PointNet [65.89809800010927]
  In cosmology, galaxy redshift surveys resemble such a permutation invariant collection of positions in space. We employ a textitPointNet-like neural network to regress the values of the cosmological parameters directly from point cloud data. Our implementation of PointNets can analyse inputs of $mathcalO(104) - mathcalO(105)$ galaxies at a time, which improves upon earlier work for this application by roughly two orders of magnitude.
  arXiv  Detail & Related papers  (2022-11-22T15:35:05Z)
• Primordial non-Gaussianity from the Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey I: Catalogue Preparation and Systematic Mitigation [3.2855185490071444]
  We investigate the large-scale clustering of the final spectroscopic sample of quasars from the recently completed extended Baryon Oscillation Spectroscopic Survey (eBOSS) We develop a neural network-based approach to mitigate spurious fluctuations in the density field caused by spatial variations in the quality of the imaging data used to select targets for follow-up spectroscopy.
  arXiv  Detail & Related papers  (2021-06-25T16:01:19Z)
• DeepShadows: Separating Low Surface Brightness Galaxies from Artifacts using Deep Learning [70.80563014913676]
  We investigate the use of convolutional neural networks (CNNs) for the problem of separating low-surface-brightness galaxies from artifacts in survey images. We show that CNNs offer a very promising path in the quest to study the low-surface-brightness universe.
  arXiv  Detail & Related papers  (2020-11-24T22:51:08Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• Interpreting Galaxy Deblender GAN from the Discriminator's Perspective [50.12901802952574]
  This research focuses on behaviors of one of the network's major components, the Discriminator, which plays a vital role but is often overlooked. We demonstrate that our method clearly reveals attention areas of the Discriminator when differentiating generated galaxy images from ground truth images.
  arXiv  Detail & Related papers  (2020-01-17T04:05:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.