A General-Purpose VLM Can Teach an Astronomy Foundation Model to Better Recognize Galaxy Morphology

• URL: http://arxiv.org/abs/2608.02300v1
• Date: Mon, 03 Aug 2026 14:26:25 GMT
• Title: A General-Purpose VLM Can Teach an Astronomy Foundation Model to Better Recognize Galaxy Morphology
• Abstract summary: We show that VLM-based VQA systems contain meaningful visual-semantic priors that can serve as weak supervision for downstream morphology classifiers.<n>We use a general-purpose VLM as a morphology teacher for Zoobot, an astronomy foundation model pretrained on large-scale Galaxy Zoo annotations.
• Score: 40.56594396436931
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Existing astronomy foundation models provide strong galaxy representations, but adapting them to new survey conditions and survey-specific morphology recognition tasks still requires substantial human supervision. We show that VLM-based VQA systems contain meaningful visual-semantic priors that can serve as weak supervision for downstream morphology classifiers and improve morphology classification under limited human-label budgets. We first introduce a survey-oriented VQA benchmark spanning two representative imaging regimes and evaluate state-of-the-art VLMs on galaxy morphology questions. The results show that these models capture useful morphology signals and informative uncertainty, but are not sufficiently reliable to replace human annotators. Motivated by this finding, we use a general-purpose VLM as a morphology teacher for Zoobot, an astronomy foundation model pretrained on large-scale Galaxy Zoo annotations. Across two survey domains and multiple annotation budgets, the VLM teacher consistently improves Zoobot's downstream morphology classification. These results demonstrate that a general-purpose VLM provides knowledge complementary to an astronomy foundation model and can teach it to better recognize galaxy morphology under limited human supervision. The resulting pipeline is designed for label-efficient adaptation to forthcoming large-scale surveys, including the Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) and the Nancy Grace Roman Space Telescope. The benchmark and code are publicly available at https://github.com/fw-ic/VLM-morphology-teacher.

Related papers

• PhysMLLMs: Spatial Priors for Unified Referring Segmentation and Grounded Reasoning of Images and Videos [116.62982600141608]
  Video large language models support language guided video segmentation, but they often show temporal inconsistencies.<n>We present PhysMLLMs, a training-stage prior injection architecture that injects physics-inspired spatial continuity priors into Video MLLMs.<n>Across multiple video benchmarks, PhysMLLMs improves video segmentation mask quality and cross-frame consistency.
  arXiv  Detail & Related papers  (2026-08-25T14:01:59Z)
• TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
  Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
  arXiv  Detail & Related papers  (2026-03-03T13:28:07Z)
• Radio Astronomy in the Era of Vision-Language Models: Prompt Sensitivity and Adaptation [5.711705587813085]
  Vision-Language Models (VLMs) are positioned as general-purpose AI systems capable of reasoning across domains.<n>We assess whether generic VLMs, presumed to lack exposure to astronomical corpora, can perform morphology-based classification of radio galaxies.
  arXiv  Detail & Related papers  (2025-08-31T14:31:47Z)
• Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation [4.3933321767775135]
  We propose a conditional diffusion model to synthesize realistic galaxy images for augmenting machine learning data.<n>We show that our model generates diverse, high-fidelity galaxy images closely adhere to the specified morphological feature conditions.<n>This model enables generative extrapolation to project well-annotated data into unseen domains and advancing rare object detection.
  arXiv  Detail & Related papers  (2025-06-19T11:44:09Z)
• TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation [65.74990259650984]
  We introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery.<n>Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism.<n>TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench.
  arXiv  Detail & Related papers  (2025-06-06T17:59:50Z)
• Spiral-Elliptical automated galaxy morphology classification from telescope images [0.40792653193642503]
  We develop two novel galaxy morphology statistics, descent average and descent variance, which can be efficiently extracted from telescope galaxy images. We utilize the galaxy image data from the Sloan Digital Sky Survey to demonstrate the effective performance of our proposed image statistics.
  arXiv  Detail & Related papers  (2023-10-10T22:36:52Z)
• Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection [57.85347204640585]
  We develop a Universal Domain Adaptation method DeepAstroUDA. It can be applied to datasets with different types of class overlap. For the first time, we demonstrate the successful use of domain adaptation on two very different observational datasets.
  arXiv  Detail & Related papers  (2022-11-01T18:07:21Z)
• Morphological classification of astronomical images with limited labelling [0.0]
  We propose an effective semi-supervised approach for galaxy morphology classification task, based on active learning of adversarial autoencoder (AAE) model. For a binary classification problem (top level question of Galaxy Zoo 2 decision tree) we achieved accuracy 93.1% on the test part with only 0.86 millions markup actions. Our best model with additional markup accuracy of 95.5%.
  arXiv  Detail & Related papers  (2021-04-27T19:26:27Z)
• Task-Agnostic Morphology Evolution [94.97384298872286]
  Current approaches that co-adapt morphology and behavior use a specific task's reward as a signal for morphology optimization. This often requires expensive policy optimization and results in task-dependent morphologies that are not built to generalize. We propose a new approach, Task-Agnostic Morphology Evolution (TAME), to alleviate both of these issues.
  arXiv  Detail & Related papers  (2021-02-25T18:59:21Z)
• DeepShadows: Separating Low Surface Brightness Galaxies from Artifacts using Deep Learning [70.80563014913676]
  We investigate the use of convolutional neural networks (CNNs) for the problem of separating low-surface-brightness galaxies from artifacts in survey images. We show that CNNs offer a very promising path in the quest to study the low-surface-brightness universe.
  arXiv  Detail & Related papers  (2020-11-24T22:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.