Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference

• URL: http://arxiv.org/abs/2608.26354v1
• Date: Wed, 26 Aug 2026 19:37:18 GMT
• Title: Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference
• Abstract summary: We show that a self-supervised Vision Transformer (ViT), pretrained label-free on a fast approximate simulator, produces transferable data summaries that generalize across simulators.<n>Without retraining, it can be reused as a frozen encoder to infer astrophysical parameters from a completely different simulator.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simulation-based inference (SBI) for parameter estimation is vulnerable to model misspecification: neural summaries and density estimators trained on a specific forward model typically fail when applied to data drawn from another model, or from real observations, and no training simulator can capture the full observational pipeline of a real measurement exactly. We show that a self-supervised Vision Transformer (ViT), pretrained label-free on a fast approximate simulator, produces transferable data summaries that generalize across simulators. Without retraining, it can be reused as a frozen encoder to infer astrophysical parameters from a completely different simulator that resolves the radiative transfer explicitly, on which it has never seen either data or parameters. As a concrete use case in 21cm cosmology, SKATR, a ViT pretrained with a Joint Embedding Predictive Architecture (JEPA), serves as a foundation model for reionization inference from upcoming SKA measurements: SKATR is pretrained once on 67k low-cost, noiseless semi-numerical 21cmFAST lightcones, then frozen and applied to hydrodynamical Loreli II lightcones, where a lightweight conditional flow matching head infers five astrophysical parameters; the encoder is never shown Loreli data, its parameters, or any noise. In our comparison, SKATR yields the most precise and best-calibrated posteriors across all five parameters, matching the accuracy of the fully-supervised in-domain baseline while requiring 2.6x fewer radiative-transfer simulations. Under realistic SKA AA* noise, only SKATR remains simultaneously accurate, informative, and calibrated, outperforming even a supervised baseline retrained from scratch on noisy data. Self-supervised pretraining on computationally efficient semi-numerical simulations is therefore a viable route to calibrated, simulator- and noise-agnostic reionization inference for the SKA-era.

Related papers

• Task-Relevant Feature-Dynamics Fidelity Enables Zero-Shot Sim-to-Real Transfer for Robotic Ultrasound Scanning [58.25007580703646]
  We develop a TR-oriented ultrasound simulator that combines a shared structural intermediate domain, trajectory-level fixed noise, and few-step conditional flow generation.<n>In phantom experiments, a policy trained exclusively in simulation succeeded in 390 of 400 zero-shot deployments across four target planes.
  arXiv  Detail & Related papers  (2026-08-30T02:37:00Z)
• mmSimPrior: Learning Simulation Priors for Data-Efficient and Generalizable Real-World Radar-based Human Motion Reconstruction [12.731621912265986]
  We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors.<n>Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains.
  arXiv  Detail & Related papers  (2026-07-25T00:56:44Z)
• Neural Posterior Estimation of Terrain Parameters from Radar Sounder Data [49.62628745057007]
  We propose a simulation-based inference approach to terrain parameter inversion from radar sounder data.<n>Synthetic observations from a GPU-based simulator are used to train a neural network-based density estimator for neural posterior estimation.<n>We demonstrate that our NPE model is well calibrated on simulated data and transferable to real Mars radar profiles.
  arXiv  Detail & Related papers  (2026-05-05T08:41:02Z)
• Multi-Fidelity Emulation of Atmospheric Correction Coefficients with Physics-Guided Kolmogorov-Arnold Networks [0.17188280334580194]
  Atmospheric correction is a critical preprocessing step in optical remote sensing.<n>This study presents a physics-aware multi-fidelity surrogate framework for emulating atmospheric correction coefficients.
  arXiv  Detail & Related papers  (2026-05-04T22:43:25Z)
• Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking [51.56484100374058]
  We evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods.<n>Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models.
  arXiv  Detail & Related papers  (2026-02-03T16:01:22Z)
• Flow Matching for Robust Simulation-Based Inference under Model Misspecification [11.172752919335394]
  Flow Matching Corrected Posterior Estimation is a framework that refines simulation-trained posterior estimators using a small set of real calibration samples.<n>We show that our proposal consistently mitigates the effects of misspecification, delivering improved inference accuracy and uncertainty calibration compared to standard SBI baselines.
  arXiv  Detail & Related papers  (2025-09-27T16:10:53Z)
• OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction [62.385417528148224]
  We present OmniTraj, a Transformer-based model pre-trained on a large-scale, heterogeneous dataset.<n>Experiments show that explicitly conditioning on the frame rate enables OmniTraj to achieve state-of-the-art zero-shot transfer performance.
  arXiv  Detail & Related papers  (2025-07-31T15:37:09Z)
• G-Sim: Generative Simulations with Large Language Models and Gradient-Free Calibration [48.948187359727996]
  G-Sim is a hybrid framework that automates simulator construction with rigorous empirical calibration.<n>It produces reliable, causally-informed simulators, mitigating data-inefficiency and enabling robust system-level interventions.
  arXiv  Detail & Related papers  (2025-06-10T22:14:34Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Sim2Real for Environmental Neural Processes [20.850715955359593]
  We analyse 'Sim2Real': pre-training on reanalysis and fine-tuning on observational data. Sim2Real could enable more accurate models for weather prediction and climate monitoring.
  arXiv  Detail & Related papers  (2023-10-30T18:49:06Z)
• PCGen: Point Cloud Generator for LiDAR Simulation [10.692184635629792]
  Existing methods generate data which are more noisy and complete than the real point clouds. We propose FPA raycasting and surrogate model raydrop. With minimal training data, the surrogate model can generalize to different geographies and scenes. Results show that object detection models trained by simulation data can achieve similar result as the real data trained model.
  arXiv  Detail & Related papers  (2022-10-17T04:13:21Z)
• AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
  We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective. We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
  arXiv  Detail & Related papers  (2020-08-16T11:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.