GalaxAlign: Mimicking Citizen Scientists' Multimodal Guidance for Galaxy Morphology Analysis

• URL: http://arxiv.org/abs/2411.19475v2
• Date: Wed, 03 Sep 2025 09:19:05 GMT
• Title: GalaxAlign: Mimicking Citizen Scientists' Multimodal Guidance for Galaxy Morphology Analysis
• Authors: Ruoqi Wang, Haitao Wang, Qiong Luo,
• Abstract summary: Galaxy morphology analysis involves studying galaxies based on their shapes and structures.<n>Existing methods either directly train domain-specific foundation models on large, annotated datasets or fine-tune vision foundation models on a smaller set of images.<n>We introduce GalaxAlign, a multimodal approach inspired by how citizen scientists identify galaxies in astronomical images.
• Score: 4.557098550366064
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Galaxy morphology analysis involves studying galaxies based on their shapes and structures. For such studies, fundamental tasks include identifying and classifying galaxies in astronomical images, as well as retrieving visually or structurally similar galaxies through similarity search. Existing methods either directly train domain-specific foundation models on large, annotated datasets or fine-tune vision foundation models on a smaller set of images. The former is effective but costly, while the latter is more resource-efficient but often yields lower accuracy. To address these challenges, we introduce GalaxAlign, a multimodal approach inspired by how citizen scientists identify galaxies in astronomical images by following textual descriptions and matching schematic symbols. Specifically, GalaxAlign employs a tri-modal alignment framework to align three types of data during fine-tuning: (1) schematic symbols representing galaxy shapes and structures, (2) textual labels for these symbols, and (3) galaxy images. By incorporating multimodal instructions, GalaxAlign eliminates the need for expensive pretraining and enhances the effectiveness of fine-tuning. Experiments on galaxy classification and similarity search demonstrate that our method effectively fine-tunes general pre-trained models for astronomical tasks by incorporating domain-specific multi-modal knowledge. Code is available at https://github.com/RapidsAtHKUST/GalaxAlign.

Related papers

• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling [68.14113731953971]
  This paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like imagination.<n>We show that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks.
  arXiv  Detail & Related papers  (2025-12-01T16:01:41Z)
• Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learning [64.32618490065117]
  A core problem of Embodied AI is to learn object manipulation from observation, as humans do.<n>We propose a novel approach that learns an affordance-aware 3D representation and employs a stage-wise inference strategy.<n> Experiments demonstrate the effectiveness of our method, showing improved performance in both affordance grounding and classification.
  arXiv  Detail & Related papers  (2025-08-02T04:14:18Z)
• Galaxy image simplification using Generative AI [0.0]
  We introduce a new approach to galaxy image analysis that is based on generative AI.<n>The method simplifies the galaxy images and automatically converts them into a skeletonized" form.<n>We demonstrate the method by applying it to galaxy images acquired by the DESI Legacy Survey.
  arXiv  Detail & Related papers  (2025-07-15T19:48:09Z)
• Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation [4.3933321767775135]
  We propose a conditional diffusion model to synthesize realistic galaxy images for augmenting machine learning data.<n>We show that our model generates diverse, high-fidelity galaxy images closely adhere to the specified morphological feature conditions.<n>This model enables generative extrapolation to project well-annotated data into unseen domains and advancing rare object detection.
  arXiv  Detail & Related papers  (2025-06-19T11:44:09Z)
• UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting [64.31900521467362]
  No existing pre-training method is equally effective for both object- and scene-level point clouds.<n>We introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture.
  arXiv  Detail & Related papers  (2025-06-11T17:23:21Z)
• Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications [83.8743080143778]
  A visual gyroscope estimates camera rotation through images. The integration of omnidirectional cameras, offering a larger field of view compared to traditional RGB cameras, has proven to yield more accurate and robust results. Here, we address these challenges by introducing a novel visual gyroscope, which combines an Efficient Multi-Mask-Filter Rotation Estor and a Learning based optimization.
  arXiv  Detail & Related papers  (2024-04-02T13:19:06Z)
• Galaxy Classification: A machine learning approach for classifying shapes using numerical data [0.0]
  We present a machine learning model for galaxy classification using numerical data from the Galaxy Zoo project. Our results show that our model achieves high accuracy in classifying galaxies and has the potential to significantly enhance our understanding of the formation and evolution of galaxies.
  arXiv  Detail & Related papers  (2023-11-30T20:47:16Z)
• Spiral-Elliptical automated galaxy morphology classification from telescope images [0.40792653193642503]
  We develop two novel galaxy morphology statistics, descent average and descent variance, which can be efficiently extracted from telescope galaxy images. We utilize the galaxy image data from the Sloan Digital Sky Survey to demonstrate the effective performance of our proposed image statistics.
  arXiv  Detail & Related papers  (2023-10-10T22:36:52Z)
• Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection [57.85347204640585]
  We develop a Universal Domain Adaptation method DeepAstroUDA. It can be applied to datasets with different types of class overlap. For the first time, we demonstrate the successful use of domain adaptation on two very different observational datasets.
  arXiv  Detail & Related papers  (2022-11-01T18:07:21Z)
• Inferring Structural Parameters of Low-Surface-Brightness-Galaxies with Uncertainty Quantification using Bayesian Neural Networks [70.80563014913676]
  We show that a Bayesian Neural Network (BNN) can be used for the inference, with uncertainty, of such parameters from simulated low-surface-brightness galaxy images. Compared to traditional profile-fitting methods, we show that the uncertainties obtained using BNNs are comparable in magnitude, well-calibrated, and the point estimates of the parameters are closer to the true values.
  arXiv  Detail & Related papers  (2022-07-07T17:55:26Z)
• Generic Lithography Modeling with Dual-band Optics-Inspired Neural Networks [52.200624127512874]
  We introduce a dual-band optics-inspired neural network design that considers the optical physics underlying lithography. Our approach yields the first published via/metal layer contour simulation at 1nm2/pixel resolution with any tile size. We also achieve 85X simulation speedup over traditional lithography simulator with 1% accuracy loss.
  arXiv  Detail & Related papers  (2022-03-12T08:08:50Z)
• MonoDistill: Learning Spatial Features for Monocular 3D Object Detection [80.74622486604886]
  We propose a simple and effective scheme to introduce the spatial information from LiDAR signals to the monocular 3D detectors. We use the resulting data to train a 3D detector with the same architecture as the baseline model. Experimental results show that the proposed method can significantly boost the performance of the baseline model.
  arXiv  Detail & Related papers  (2022-01-26T09:21:41Z)
• SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations [85.38562724999898]
  We propose a 2D Image and 3D Point cloud Unsupervised pre-training strategy, called SimIPU. Specifically, we develop a multi-modal contrastive learning framework that consists of an intra-modal spatial perception module and an inter-modal feature interaction module. To the best of our knowledge, this is the first study to explore contrastive learning pre-training strategies for outdoor multi-modal datasets.
  arXiv  Detail & Related papers  (2021-12-09T03:27:00Z)
• Practical Galaxy Morphology Tools from Deep Supervised Representation Learning [0.0]
  We show that deep learning models trained to answer every Galaxy Zoo DECaLS question learn meaningful semantic representations of galaxies. We exploit these representations to outperform existing approaches at several practical tasks crucial for investigating large galaxy samples.
  arXiv  Detail & Related papers  (2021-10-25T08:46:16Z)
• Morphological Classification of Galaxies in S-PLUS using an Ensemble of Convolutional Networks [0.0]
  We combine accurate visual classifications of the Galaxy Zoo project with emph Deep Learning methods. A neural network model was created through an Ensemble of four other convolutional models, allowing a greater accuracy in the classification than what would be obtained with any one individual.
  arXiv  Detail & Related papers  (2021-07-05T21:51:19Z)
• Morphological classification of astronomical images with limited labelling [0.0]
  We propose an effective semi-supervised approach for galaxy morphology classification task, based on active learning of adversarial autoencoder (AAE) model. For a binary classification problem (top level question of Galaxy Zoo 2 decision tree) we achieved accuracy 93.1% on the test part with only 0.86 millions markup actions. Our best model with additional markup accuracy of 95.5%.
  arXiv  Detail & Related papers  (2021-04-27T19:26:27Z)
• Deep Magnification-Flexible Upsampling over 3D Point Clouds [103.09504572409449]
  We propose a novel end-to-end learning-based framework to generate dense point clouds. We first formulate the problem explicitly, which boils down to determining the weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted weights as well as the high-order refinements.
  arXiv  Detail & Related papers  (2020-11-25T14:00:18Z)
• DeepShadows: Separating Low Surface Brightness Galaxies from Artifacts using Deep Learning [70.80563014913676]
  We investigate the use of convolutional neural networks (CNNs) for the problem of separating low-surface-brightness galaxies from artifacts in survey images. We show that CNNs offer a very promising path in the quest to study the low-surface-brightness universe.
  arXiv  Detail & Related papers  (2020-11-24T22:51:08Z)
• Interpreting Galaxy Deblender GAN from the Discriminator's Perspective [50.12901802952574]
  This research focuses on behaviors of one of the network's major components, the Discriminator, which plays a vital role but is often overlooked. We demonstrate that our method clearly reveals attention areas of the Discriminator when differentiating generated galaxy images from ground truth images.
  arXiv  Detail & Related papers  (2020-01-17T04:05:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.