Astroconformer: The Prospects of Analyzing Stellar Light Curves with Transformer-Based Deep Learning Models

• URL: http://arxiv.org/abs/2309.16316v2
• Date: Thu, 18 Jan 2024 10:23:17 GMT
• Title: Astroconformer: The Prospects of Analyzing Stellar Light Curves with Transformer-Based Deep Learning Models
• Authors: Jia-Shu Pan, Yuan-Sen Ting, Jie Yu
• Abstract summary: We introduce $textitAstroconformer$, a Transformer-based deep learning framework, specifically designed to capture long-range dependencies in stellar light curves. $textitAstroconformer$ demonstrates superior performance, achieving a root-mean-square-error (RMSE) of 0.017 dex at $log gapprox3$ in data-rich regimes. $textitAstroconformer$ also excels in extracting $nu_max$ with high precision.
• Score: 2.9203802343391057
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stellar light curves contain valuable information about oscillations and granulation, offering insights into stars' internal structures and evolutionary states. Traditional asteroseismic techniques, primarily focused on power spectral analysis, often overlook the crucial phase information in these light curves. Addressing this gap, recent machine learning applications, particularly those using Convolutional Neural Networks (CNNs), have made strides in inferring stellar properties from light curves. However, CNNs are limited by their localized feature extraction capabilities. In response, we introduce $\textit{Astroconformer}$, a Transformer-based deep learning framework, specifically designed to capture long-range dependencies in stellar light curves. Our empirical analysis centers on estimating surface gravity ($\log g$), using a dataset derived from single-quarter Kepler light curves with $\log g$ values ranging from 0.2 to 4.4. $\textit{Astroconformer}$ demonstrates superior performance, achieving a root-mean-square-error (RMSE) of 0.017 dex at $\log g\approx3$ in data-rich regimes and up to 0.1 dex in sparser areas. This performance surpasses both K-nearest neighbor models and advanced CNNs. Ablation studies highlight the influence of receptive field size on model effectiveness, with larger fields correlating to improved results. $\textit{Astroconformer}$ also excels in extracting $\nu_{\max}$ with high precision. It achieves less than 2% relative median absolute error for 90-day red giant light curves. Notably, the error remains under 3% for 30-day light curves, whose oscillations are undetectable by a conventional pipeline in 30% cases. Furthermore, the attention mechanisms in $\textit{Astroconformer}$ align closely with the characteristics of stellar oscillations and granulation observed in light curves.

Related papers

• The Scaling Law in Stellar Light Curves [3.090476527764192]
  We investigate the scaling law properties that emerge when learning from astronomical time series data using self-supervised techniques. A self-supervised Transformer model achieves 3-10 times the sample efficiency compared to the state-of-the-art supervised learning model. Our research lays the groundwork for analyzing stellar light curves by examining them through large-scale auto-regressive generative models.
  arXiv  Detail & Related papers  (2024-05-27T13:31:03Z)
• Bayesian Inference with Deep Weakly Nonlinear Networks [57.95116787699412]
  We show at a physics level of rigor that Bayesian inference with a fully connected neural network is solvable. We provide techniques to compute the model evidence and posterior to arbitrary order in $1/N$ and at arbitrary temperature.
  arXiv  Detail & Related papers  (2024-05-26T17:08:04Z)
• Learning Galaxy Intrinsic Alignment Correlations [16.805775045014578]
  The intrinsic alignments (IA) of galaxies represent the correlation of galaxy shapes due to gravitational tidal interactions and galaxy formation processes. We present a deep learning approach to emulate galaxy position-position, position-orientation, and orientation-orientation correlation function measurements and uncertainties.
  arXiv  Detail & Related papers  (2024-04-21T16:16:56Z)
• One-dimensional Convolutional Neural Networks for Detecting Transiting Exoplanets [39.58317527488534]
  We develop an artificial neural network model that is able to detect transits in light curves obtained from different telescopes and surveys. We created artificial light curves with and without transits to try to mimic those expected for the extended mission of the Kepler telescope (K2) in order to train and validate a 1D convolutional neural network model.
  arXiv  Detail & Related papers  (2023-12-12T10:56:27Z)
• Local primordial non-Gaussianity from the large-scale clustering of photometric DESI luminous red galaxies [5.534428269834764]
  We use angular clustering of luminous red galaxies from the Dark Energy Spectroscopic Instrument (DESI) imaging surveys to constrain the local primordial non-Gaussianity parameter $fnl$. Our sample comprises over 12 million targets, covering 14,000 square degrees of the sky, with redshifts in the range 0.2 z 1.35$. We identify Galactic extinction, survey depth, and astronomical seeing as the primary sources of systematic error, and employ linear regression and artificial neural networks to alleviate non-cosmological excess clustering on large scales.
  arXiv  Detail & Related papers  (2023-07-04T14:49:23Z)
• Neural Inference of Gaussian Processes for Time Series Data of Quasars [72.79083473275742]
  We introduce a new model that enables it to describe quasar spectra completely. We also introduce a new method of inference of Gaussian process parameters, which we call $textitNeural Inference$. The combination of both the CDRW model and Neural Inference significantly outperforms the baseline DRW and MLE.
  arXiv  Detail & Related papers  (2022-11-17T13:01:26Z)
• Light curve completion and forecasting using fast and scalable Gaussian processes (MuyGPs) [0.0]
  Ground-based observations from commercial off the shelf (COTS) cameras remain inexpensive compared to higher precision instruments. limited sensor availability combined with noisier observations can produce gappy time-series data. Deep Neural Networks (DNNs) have become the tool of choice due to their empirical success at learning complex nonlinear embeddings.
  arXiv  Detail & Related papers  (2022-08-31T01:52:00Z)
• Supernova Light Curves Approximation based on Neural Network Models [53.180678723280145]
  Photometric data-driven classification of supernovae becomes a challenge due to the appearance of real-time processing of big data in astronomy. Recent studies have demonstrated the superior quality of solutions based on various machine learning models. We study the application of multilayer perceptron (MLP), bayesian neural network (BNN), and normalizing flows (NF) to approximate observations for a single light curve.
  arXiv  Detail & Related papers  (2022-06-27T13:46:51Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)
• A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent [85.77233010209368]
  This article characterizes the exacts of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples $n$ is all large and comparable. This analysis also provides accurate estimates of training and test regression errors for large $n,p,N$.
  arXiv  Detail & Related papers  (2020-06-09T02:05:40Z)
• Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nystr\"om method [76.73096213472897]
  We develop techniques which exploit spectral properties of the data matrix to obtain improved approximation guarantees. Our approach leads to significantly better bounds for datasets with known rates of singular value decay. We show that both our improved bounds and the multiple-descent curve can be observed on real datasets simply by varying the RBF parameter.
  arXiv  Detail & Related papers  (2020-02-21T00:43:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.