A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation

• URL: http://arxiv.org/abs/2607.27501v1
• Date: Wed, 29 Jul 2026 22:35:13 GMT
• Title: A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation
• Abstract summary: We present a lightweight approach to foundation modeling that leverages pre-trained learning from collider physics data.<n>The model pre-trains with no supervision over a large-scale collision dataset from the Large Hadron Collider.<n>Downstream tasks for collider analyses, such as kinematic regression and event classification, are developed on pre-trained model weights.
• Score: 44.18742604961034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a lightweight approach to foundation modeling (\textbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scientific datasets, using a fully connected autoencoder model with approximately 3 million parameters. The model pre-trains with no supervision over a large-scale collision dataset from the Large Hadron Collider modeled by charged particle track features. Downstream tasks for collider analyses, such as kinematic regression and event classification, are developed on pre-trained model weights and achieve improved accuracy with only small labeled datasets when compared to equivalent architectures trained from scratch. The benefits of pre-training are additionally investigated through latent space interpretation and application to other domains, including gravitational waves, flood forecasting, and neural activity. Furthermore, the relative computational simplicity of NEXUS is demonstrated compared to transformer approaches at comparable scale, opening the door to power-efficient inference and real-time or edge applications of foundation models in scientific experiments.

Related papers

• Position: Weight Space Should Be a First-Class Generative AI Modality [67.10702320971893]
  This paper argues that model checkpoints should be treated as a first-class data modality.<n>It argues that generative modeling in weight space should be standardized as a core machine learning primitive.<n>Our goal is to shift the community's default mindset from optimizing models per task to sampling models from learned weight distributions.
  arXiv  Detail & Related papers  (2026-05-18T16:38:26Z)
• Few-Shot Physics-Informed Neural Network for Shape Reconstruction of Concentric-Tube Robots [4.539331900295919]
  We present a physics-informed neural network (PINN) for kinematic modeling of a 6-DoF CTR with three pre-curved tubes.<n>PINN enables full-state estimation of shape, twist angle, torsional strain, bending moment, and orientation.<n>The resulting model is also computationally efficient and robust, making it well-suited for real-time control applications.
  arXiv  Detail & Related papers  (2026-05-12T22:07:53Z)
• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders [1.988691274281547]
  We demonstrate transfer learning capabilities in a machine-learned algorithm trained for particle-flow reconstruction in high energy particle colliders.<n>To our knowledge, this is the first full-simulation cross-detector transfer learning study for particle-flow reconstruction.
  arXiv  Detail & Related papers  (2025-02-28T19:16:01Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Foundation Models for Generalist Geospatial Artificial Intelligence [3.7002058945990415]
  This paper introduces a first-of-a-kind framework for the efficient pre-training and fine-tuning of foundational models on extensive data. We have utilized this framework to create Prithvi, a transformer-based foundational model pre-trained on more than 1TB of multispectral satellite imagery.
  arXiv  Detail & Related papers  (2023-10-28T10:19:55Z)
• A Physics Enhanced Residual Learning (PERL) Framework for Vehicle Trajectory Prediction [5.7215490229343535]
  PERL integrates the strengths of physics-based and data-driven methods for traffic state prediction. It preserves the interpretability inherent to physics-based models and has reduced data requirements. PERL achieves better prediction with a small dataset, compared to the physics model, data-driven model, and PINN model.
  arXiv  Detail & Related papers  (2023-09-26T21:41:45Z)
• Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors [1.4609888393206634]
  We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. We compare a graph neural network and kernel-based transformer and demonstrate that we can avoid operations while achieving realistic reconstruction. The best graph neural network model shows improvement in the jet transverse momentum resolution by up to 50% compared to the rule-based algorithm.
  arXiv  Detail & Related papers  (2023-09-13T08:16:15Z)
• Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior [32.74388989649232]
  We study how pre-training could be used for scientific machine learning (SciML) applications. We find that fine-tuning these models yields more performance gains as model size increases.
  arXiv  Detail & Related papers  (2023-06-01T00:32:59Z)
• Transforming Model Prediction for Tracking [109.08417327309937]
  Transformers capture global relations with little inductive bias, allowing it to learn the prediction of more powerful target models. We train the proposed tracker end-to-end and validate its performance by conducting comprehensive experiments on multiple tracking datasets. Our tracker sets a new state of the art on three benchmarks, achieving an AUC of 68.5% on the challenging LaSOT dataset.
  arXiv  Detail & Related papers  (2022-03-21T17:59:40Z)
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling [86.9726984929758]
  We focus on the integration of incomplete physics models into deep generative models. We propose a VAE architecture in which a part of the latent space is grounded by physics. We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-25T20:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.