Transfer Learning Beyond the Standard Model

• URL: http://arxiv.org/abs/2510.19168v1
• Date: Wed, 22 Oct 2025 01:59:38 GMT
• Title: Transfer Learning Beyond the Standard Model
• Abstract summary: Transfer learning offers a way to reduce the simulation cost by reusing knowledge across models.<n>We show that pre-training on the standard model of cosmology, $Lambda$CDM, can enable inference with significantly fewer beyond-$Lambda$CDM simulations.
• Score: 0.11666234644810893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning enables powerful cosmological inference but typically requires many high-fidelity simulations covering many cosmological models. Transfer learning offers a way to reduce the simulation cost by reusing knowledge across models. We show that pre-training on the standard model of cosmology, $\Lambda$CDM, and fine-tuning on various beyond-$\Lambda$CDM scenarios -- including massive neutrinos, modified gravity, and primordial non-Gaussianities -- can enable inference with significantly fewer beyond-$\Lambda$CDM simulations. However, we also show that negative transfer can occur when strong physical degeneracies exist between $\Lambda$CDM and beyond-$\Lambda$CDM parameters. We consider various transfer architectures, finding that including bottleneck structures provides the best performance. Our findings illustrate the opportunities and pitfalls of foundation-model approaches in physics: pre-training can accelerate inference, but may also hinder learning new physics.

Related papers

• Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions [0.4062071796326536]
  Future AI-based studies in particle physics will likely start from a foundation model to accelerate training and enhance sensitivity.<n>We investigate whether the OmniLearned foundation model pre-trained on diverse high-$Q2$ simulated and real collisions can be effectively transferred to a few-GeV fixed-target neutrino experiment.
  arXiv  Detail & Related papers  (2026-04-14T06:53:22Z)
• How to Set the Learning Rate for Large-Scale Pre-training? [73.03133634525635]
  We formalize this investigation into two distinct research paradigms: Fitting and Transfer.<n>Within the Fitting Paradigm, we introduce a Scaling Law for search factor, effectively reducing the search complexity from O(n3) to O(n*C_D*C_) via predictive modeling.<n>We extend the principles of $$Transfer to the Mixture of Experts (MoE) architecture, broadening its applicability to encompass model depth, weight decay, and token horizons.
  arXiv  Detail & Related papers  (2026-01-08T15:55:13Z)
• Latent Representation Learning of Multi-scale Thermophysics: Application to Dynamics in Shocked Porous Energetic Material [2.5950614721140832]
  We propose an alternative meta-learning approach motivated by the idea of tokenization in natural language processing.<n>We show that one can learn a reduced representation of the micro-scale physics to accelerate the meso-scale learning process.<n>The proposed approach accelerates the development of closure models by leveraging inexpensive micro-scale simulations and fast training over a small meso-scale dataset.
  arXiv  Detail & Related papers  (2025-06-15T23:28:33Z)
• Transfer learning for multifidelity simulation-based inference in cosmology [0.0]
  Pre-training on dark-matter-only $N$-body simulations reduces the required number of high-fidelity hydrodynamical simulations by a factor between $8$ and $15$.<n>By leveraging cheaper simulations, our approach enables performant and accurate inference on high-fidelity models while substantially reducing computational costs.
  arXiv  Detail & Related papers  (2025-05-27T14:04:30Z)
• Flow Matching for Geometric Trajectory Simulation [4.271235935891555]
  N-body systems are a fundamental problem with applications in a wide range of fields, such as molecular dynamics, biochemistry, and pedestrian dynamics.<n>Machine learning has become an invaluable tool for scaling physics-based simulators and developing models directly from experimental data.<n>To generate realistic trajectories, existing methods must learn complex transformations starting from uninformed noise and do not allow for the exploitation of domain-informed priors.<n>We propose STFlow to address this limitation. By leveraging flow matching and data-dependent couplings, STFlow facilitates physics-informed simulation of geometric trajectories without sacrificing model expressivity or scalability.
  arXiv  Detail & Related papers  (2025-05-24T11:18:59Z)
• Learning the Universe: Learning to Optimize Cosmic Initial Conditions with Non-Differentiable Structure Formation Models [0.22940141855172033]
  We introduce Learning the Universe by Learning to Optimize (LULO) for reconstructing the 3D cosmic initial conditions.<n>LULO is a gradient-free framework for reconstructing the 3D cosmic initial conditions.<n>We demonstrate the method by accurately reconstructing initial conditions from $M_200mathrmc$ halos identified in a dark matter-only $N$-body simulation.
  arXiv  Detail & Related papers  (2025-02-18T19:17:01Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Automated Dissipation Control for Turbulence Simulation with Shell Models [1.675857332621569]
  The application of machine learning (ML) techniques, especially neural networks, has seen tremendous success at processing images and language. In this work we construct a strongly simplified representation of turbulence by using the Gledzer-Ohkitani-Yamada shell model. We propose an approach that aims to reconstruct statistical properties of turbulence such as the self-similar inertial-range scaling.
  arXiv  Detail & Related papers  (2022-01-07T15:03:52Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)
• Satellite galaxy abundance dependency on cosmology in Magneticum simulations [101.18253437732933]
  We build an emulator of satellite abundance based on cosmological parameters. We find that $A$ and $beta$ depend on cosmological parameters, even if weakly. We also show that satellite abundance cosmology dependency differs between full-physics (FP) simulations, dark-matter only (DMO) and non-radiative simulations.
  arXiv  Detail & Related papers  (2021-10-11T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.