What AstroPT knows about galaxies, and what that can teach us about LLMs

• URL: http://arxiv.org/abs/2608.22614v1
• Date: Sun, 23 Aug 2026 21:40:26 GMT
• Title: What AstroPT knows about galaxies, and what that can teach us about LLMs
• Abstract summary: Interpretability research increasingly asks when concepts emerge during training and whether linear probes recover real structure.<n>We propose the use of astronomical ground truth through AstroPT, a transformer trained on millions of galaxy images.
• Score: 2.4906335493319207
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Interpretability research increasingly asks when concepts emerge during training and whether linear probes recover real structure, but in language models these claims are hard to validate because language offers little ground-truth ordering of concepts or relationships among them. We propose the use of astronomical ground truth through AstroPT, a transformer trained on millions of galaxy images, as a calibration testbed. AstroPT is an LLM-like model trained within a domain where the difficulty ordering of concepts and the relations among them are known in advance. Probing frozen representations across checkpoints, layers, model sizes, and objective choices, we find that galaxy properties emerge in a fixed order that tracks their known difficulty---quantities written almost directly into the pixels (band magnitude) become decodable early in training and shallow in the network, while multiband/spectra based and inferred quantities (such as redshift and specific star formation rate) emerge later and deeper. This order is invariant to our tested training objectives, and scales in magnitude but not in sequence with capacity. Our linear probe directions further recover the known physical structure among galaxy properties. Our findings suggest that astronomy offers a controlled sandbox for calibrating mechanistic interpretability methods we otherwise apply to LLMs blind.

Related papers

• Galactification: painting galaxies onto dark matter only simulations using a transformer-based model [1.0614155147784068]
  We develop a framework to rapidly generate mock galaxy catalogs conditioned on inexpensive dark-matter-only simulations.<n>We present a multi-modal, transformer-based model that takes 3D dark matter density and velocity fields as input, and outputs a corresponding point cloud of galaxies.<n>We demonstrate that our trained model faithfully reproduces a variety of galaxy summary statistics and correctly captures their variation with changes in the underlying cosmological and astrophysical parameters.
  arXiv  Detail & Related papers  (2025-11-11T16:39:47Z)
• Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling [56.26834106704781]
  Factual incorrectness in generated content is one of the primary concerns in ubiquitous deployment of large language models (LLMs)<n>We provide evidence supporting the presence of LLMs' internal compass that dictate the correctness of factual recall at the time of generation.<n>Scaling experiments across model sizes and training dynamics highlight that self-awareness emerges rapidly during training and peaks in intermediate layers.
  arXiv  Detail & Related papers  (2025-05-27T16:24:02Z)
• Insights on Galaxy Evolution from Interpretable Sparse Feature Networks [0.0]
  We present a novel neural network architecture called a Sparse Feature Network (SFNet)<n>SFNets produce interpretable features that can be linearly combined in order to estimate galaxy properties like optical emission line ratios or gas-phase metallicity.<n>We find that SFNets do not sacrifice accuracy in order to gain interpretability, and that they perform comparably well to cutting-edge models on astronomical machine learning tasks.
  arXiv  Detail & Related papers  (2024-12-30T19:00:00Z)
• A Theoretical Understanding of Self-Correction through In-context Alignment [51.622068973630796]
  Large language models (LLMs) are capable of improving their abilities purely by self-correction. We show that when LLMs give relatively accurate self-examinations as rewards, they are capable of refining responses in an in-context way. Inspired by these findings, we also illustrate applications of self-correction, such as defending against LLM jailbreaks.
  arXiv  Detail & Related papers  (2024-05-28T22:33:02Z)
• LtU-ILI: An All-in-One Framework for Implicit Inference in Astrophysics and Cosmology [1.5070941464775514]
  This paper presents the Learning the Universe Implicit Likelihood Inference (LtU-ILI) pipeline. It is a cutting-edge machine learning (ML) inference in astrophysics and cosmology. We present real applications across a range of astrophysics and cosmology problems.
  arXiv  Detail & Related papers  (2024-02-06T19:00:00Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)
• Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection [57.85347204640585]
  We develop a Universal Domain Adaptation method DeepAstroUDA. It can be applied to datasets with different types of class overlap. For the first time, we demonstrate the successful use of domain adaptation on two very different observational datasets.
  arXiv  Detail & Related papers  (2022-11-01T18:07:21Z)
• Inferring Structural Parameters of Low-Surface-Brightness-Galaxies with Uncertainty Quantification using Bayesian Neural Networks [70.80563014913676]
  We show that a Bayesian Neural Network (BNN) can be used for the inference, with uncertainty, of such parameters from simulated low-surface-brightness galaxy images. Compared to traditional profile-fitting methods, we show that the uncertainties obtained using BNNs are comparable in magnitude, well-calibrated, and the point estimates of the parameters are closer to the true values.
  arXiv  Detail & Related papers  (2022-07-07T17:55:26Z)
• Learning cosmology and clustering with cosmic graphs [0.0]
  We train deep learning models on thousands of galaxy catalogues from the state-of-the-art hydrodynamic simulations of the CAMELS project. We first show that GNNs can learn to compute the power spectrum of galaxy catalogues with a few percent accuracy. We then train GNNs to perform likelihood-free inference at the galaxy-field level.
  arXiv  Detail & Related papers  (2022-04-28T18:00:02Z)
• Morphological Classification of Galaxies in S-PLUS using an Ensemble of Convolutional Networks [0.0]
  We combine accurate visual classifications of the Galaxy Zoo project with emph Deep Learning methods. A neural network model was created through an Ensemble of four other convolutional models, allowing a greater accuracy in the classification than what would be obtained with any one individual.
  arXiv  Detail & Related papers  (2021-07-05T21:51:19Z)
• DeepShadows: Separating Low Surface Brightness Galaxies from Artifacts using Deep Learning [70.80563014913676]
  We investigate the use of convolutional neural networks (CNNs) for the problem of separating low-surface-brightness galaxies from artifacts in survey images. We show that CNNs offer a very promising path in the quest to study the low-surface-brightness universe.
  arXiv  Detail & Related papers  (2020-11-24T22:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.