A Versatile Framework for Analyzing Galaxy Image Data by Implanting Human-in-the-loop on a Large Vision Model

• URL: http://arxiv.org/abs/2405.10890v1
• Date: Fri, 17 May 2024 16:29:27 GMT
• Title: A Versatile Framework for Analyzing Galaxy Image Data by Implanting Human-in-the-loop on a Large Vision Model
• Authors: Mingxiang Fu, Yu Song, Jiameng Lv, Liang Cao, Peng Jia, Nan Li, Xiangru Li, Jifeng Liu, A-Li Luo, Bo Qiu, Shiyin Shen, Liangping Tu, Lili Wang, Shoulin Wei, Haifeng Yang, Zhenping Yi, Zhiqiang Zou,
• Abstract summary: We present a framework for general analysis of galaxy images based on a large vision model (LVM) plus downstream tasks (DST) Considering the low signal-to-noise ratio of galaxy images, we have incorporated a Human-in-the-loop (HITL) module into our large vision model. For object detection, trained by 1000 data points, our DST upon the LVM achieves an accuracy of 96.7%, while ResNet50 plus Mask R-CNN gives an accuracy of 93.1%.
• Score: 14.609681101463334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The exponential growth of astronomical datasets provides an unprecedented opportunity for humans to gain insight into the Universe. However, effectively analyzing this vast amount of data poses a significant challenge. Astronomers are turning to deep learning techniques to address this, but the methods are limited by their specific training sets, leading to considerable duplicate workloads too. Hence, as an example to present how to overcome the issue, we built a framework for general analysis of galaxy images, based on a large vision model (LVM) plus downstream tasks (DST), including galaxy morphological classification, image restoration, object detection, parameter extraction, and more. Considering the low signal-to-noise ratio of galaxy images and the imbalanced distribution of galaxy categories, we have incorporated a Human-in-the-loop (HITL) module into our large vision model, which leverages human knowledge to enhance the reliability and interpretability of processing galaxy images interactively. The proposed framework exhibits notable few-shot learning capabilities and versatile adaptability to all the abovementioned tasks on galaxy images in the DESI legacy imaging surveys. Expressly, for object detection, trained by 1000 data points, our DST upon the LVM achieves an accuracy of 96.7%, while ResNet50 plus Mask R-CNN gives an accuracy of 93.1%; for morphology classification, to obtain AUC ~0.9, LVM plus DST and HITL only requests 1/50 training sets compared to ResNet18. Expectedly, multimodal data can be integrated similarly, which opens up possibilities for conducting joint analyses with datasets spanning diverse domains in the era of multi-message astronomy.

Related papers

• Preliminary Report on Mantis Shrimp: a Multi-Survey Computer Vision Photometric Redshift Model [0.431625343223275]
  Photometric redshift estimation is a well-established subfield of astronomy. Mantis Shrimp is a computer vision model for photometric redshift estimation that fuses ultra-violet (GALEX), optical (PanSTARRS), and infrared (UnWISE) imagery.
  arXiv  Detail & Related papers  (2024-02-05T21:44:19Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Discovering Galaxy Features via Dataset Distillation [7.121183597915665]
  In many applications, Neural Nets (NNs) have classification performance on par or even exceeding human capacity. Here, we apply this idea to the notoriously difficult task of galaxy classification. We present a novel way to summarize and visualize prototypical galaxy morphology through the lens of neural networks.
  arXiv  Detail & Related papers  (2023-11-29T12:39:31Z)
• Spiral-Elliptical automated galaxy morphology classification from telescope images [0.40792653193642503]
  We develop two novel galaxy morphology statistics, descent average and descent variance, which can be efficiently extracted from telescope galaxy images. We utilize the galaxy image data from the Sloan Digital Sky Survey to demonstrate the effective performance of our proposed image statistics.
  arXiv  Detail & Related papers  (2023-10-10T22:36:52Z)
• On quantifying and improving realism of images generated with diffusion [50.37578424163951]
  We propose a metric, called Image Realism Score (IRS), computed from five statistical measures of a given image. IRS is easily usable as a measure to classify a given image as real or fake. We experimentally establish the model- and data-agnostic nature of the proposed IRS by successfully detecting fake images generated by Stable Diffusion Model (SDM), Dalle2, Midjourney and BigGAN. Our efforts have also led to Gen-100 dataset, which provides 1,000 samples for 100 classes generated by four high-quality models.
  arXiv  Detail & Related papers  (2023-09-26T08:32:55Z)
• Delving Deeper into Data Scaling in Masked Image Modeling [145.36501330782357]
  We conduct an empirical study on the scaling capability of masked image modeling (MIM) methods for visual recognition. Specifically, we utilize the web-collected Coyo-700M dataset. Our goal is to investigate how the performance changes on downstream tasks when scaling with different sizes of data and models.
  arXiv  Detail & Related papers  (2023-05-24T15:33:46Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model [97.9548609175831]
  We resort to plain vision transformers with about 100 million parameters and make the first attempt to propose large vision models customized for remote sensing tasks. Specifically, to handle the large image size and objects of various orientations in RS images, we propose a new rotated varied-size window attention. Experiments on detection tasks demonstrate the superiority of our model over all state-of-the-art models, achieving 81.16% mAP on the DOTA-V1.0 dataset.
  arXiv  Detail & Related papers  (2022-08-08T09:08:40Z)
• Realistic galaxy image simulation via score-based generative models [0.0]
  We show that a score-based generative model can be used to produce realistic yet fake images that mimic observations of galaxies. Subjectively, the generated galaxies are highly realistic when compared with samples from the real dataset.
  arXiv  Detail & Related papers  (2021-11-02T16:27:08Z)
• Morphological classification of astronomical images with limited labelling [0.0]
  We propose an effective semi-supervised approach for galaxy morphology classification task, based on active learning of adversarial autoencoder (AAE) model. For a binary classification problem (top level question of Galaxy Zoo 2 decision tree) we achieved accuracy 93.1% on the test part with only 0.86 millions markup actions. Our best model with additional markup accuracy of 95.5%.
  arXiv  Detail & Related papers  (2021-04-27T19:26:27Z)
• DeepShadows: Separating Low Surface Brightness Galaxies from Artifacts using Deep Learning [70.80563014913676]
  We investigate the use of convolutional neural networks (CNNs) for the problem of separating low-surface-brightness galaxies from artifacts in survey images. We show that CNNs offer a very promising path in the quest to study the low-surface-brightness universe.
  arXiv  Detail & Related papers  (2020-11-24T22:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.