Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility

• URL: http://arxiv.org/abs/2607.00228v1
• Date: Tue, 30 Jun 2026 22:15:54 GMT
• Title: Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility
• Abstract summary: We introduce the ORACLE-2 models, which combine light curves, metadata, and images for real-time hierarchical classification.<n>We show that incorporating additional modalities consistently improves classification performance.<n>These results show that combining multiple modalities improves early-time classification, enabling more effective triage of high-volume alert streams.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern time-domain surveys such as the Zwicky Transient Facility (ZTF) generate hundreds of thousands of alerts each night, making real-time decisions for follow-up observations a central challenge in time-domain astronomy. Robust early classification is crucial for making informed decisions, but is hindered by sparse light curves and degeneracies between classes. In this work, we leverage multimodality to substantially improve real-time classification and demonstrate the practicality of our approach by deploying our model on the ZTF alert stream. Building on the Online Ranked Astrophysical CLass Estimator (ORACLE), we introduce the ORACLE-2 models, which combine light curves, metadata, and images for real-time hierarchical classification. Using both real and simulated datasets, we show that incorporating additional modalities consistently improves classification performance. On observations from ZTF's Bright Transient Survey, our best-performing model, ORACLE-2 Omni, achieves a macro F1 score of 0.73 -- an improvement of up to 11% over models using light curves and metadata alone, and up to 40% over light-curve-only models, with the strongest gains realized at early times. To demonstrate applicability to the Legacy Survey of Space and Time, which will increase alert volume by more than an order of magnitude, we train a light curve + metadata variant on the simulated ELAsTiCC dataset. This model achieves a macro F1 score of 0.88, an improvement of up to 13% over the light-curve-only variant, matching the performance of other state-of-the-art models. Finally, we quantify the trade-offs between performance and throughput, identifying regimes where multimodal approaches offer the greatest benefit. These results show that combining multiple modalities improves early-time classification, enabling more effective triage of high-volume alert streams for current and future time-domain surveys.

Related papers

• Domain-Informed Multi-View Self-Distillation for Astronomical Light-Curve Representation Learning with JEPA [0.0]
  We propose a domain-informed representation learning framework for irregular astronomical time series.<n>We combine semantics-preserving views, uncertainty-aware tokenization, and multi-view self-distillation.<n>Our model outperforms hand-crafted features on 15 of 16 classification metrics.
  arXiv  Detail & Related papers  (2026-06-26T08:35:55Z)
• CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating [49.94171749437024]
  We propose Concentrate and Concentrate (CaC), a coarse-to-fine anomaly reward model based on Vision-Language Models.<n>We construct the first largescale generated video anomaly dataset with per-frame bounding-box annotations, temporal anomaly windows, and fine-grained attribution labels.<n>In experiments, CaC can stably concentrate on subtle anomalies, achieving a 25.7% accuracy improvement on fine-grained anomaly benchmarks.
  arXiv  Detail & Related papers  (2026-05-12T08:08:33Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams [0.42970700836450476]
  This paper introduces CALM, a novel, end-to-end framework for real-time anomaly detection.<n> CALM is built on the Apache Beam distributed processing framework.<n>It implements a closed-loop, continuous fine-tuning mechanism that allows the anomaly detection model to adapt to evolving data patterns in near real-time.
  arXiv  Detail & Related papers  (2025-08-29T00:27:35Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)
• Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals [49.17123504516502]
  CurrentReinforcement Fine-tuning (RFT) paradigms for Large Language Models (LLMs) suffer from inefficiency due to redundant exposure of identical queries under uniform data sampling.<n>We propose a Gradient-driven Angle-Informed Navigated RL framework.<n>By leveraging the model's intrinsic angle concentration signal, GAIN-RL dynamically selects training data in each epoch, ensuring consistently impactful gradient updates.
  arXiv  Detail & Related papers  (2025-06-02T21:40:38Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)
• AstroM$^3$: A self-supervised multimodal model for astronomy [0.0]
  We propose AstroM$3$, a self-supervised pre-training approach that enables a model to learn from multiple modalities simultaneously. Specifically, we extend the CLIP (Contrastive Language-Image Pretraining) model to a trimodal setting, allowing the integration of time-series photometry data, spectra, and astrophysical metadata. Results demonstrate that CLIP pre-training improves classification performance for time-series photometry, where accuracy increases from 84.6% to 91.5%.
  arXiv  Detail & Related papers  (2024-11-13T18:20:29Z)
• Visual Fourier Prompt Tuning [63.66866445034855]
  We propose the Visual Fourier Prompt Tuning (VFPT) method as a general and effective solution for adapting large-scale transformer-based models. Our approach incorporates the Fast Fourier Transform into prompt embeddings and harmoniously considers both spatial and frequency domain information. Our results demonstrate that our approach outperforms current state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2024-11-02T18:18:35Z)
• Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification [7.592813175419603]
  We present a comprehensive evaluation of deep-learning and large language model (LLM) based models for the automatic classification of variable star light curves.<n>Special emphasis is placed on Cepheids, RR Lyrae, and eclipsing binaries, examining the influence of observational cadence and phase distribution on classification precision.<n>We unveil StarWhisper LightCurve (LC), an innovative Series comprising three LLM-based models: LLM, multimodal large language model (MLLM), and Large Audio Language Model (LALM)
  arXiv  Detail & Related papers  (2024-04-16T17:35:25Z)
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves [0.0]
  supernovae (SNe) are relatively uncommon objects compared to other classes of variable events. processing of multi-band light-curves is a challenging task due to the highly irregular cadence, long time gaps, missing-values, low number of observations. We propose a Deep Attention model called TimeModAttn to classify multi-band light-curves of different SN types. Second, we propose a model for the synthetic generation of SN multi-band light-curves based on the Supernova Parametric Model (SPM)
  arXiv  Detail & Related papers  (2022-01-20T22:48:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.