Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training

• URL: http://arxiv.org/abs/2604.07037v1
• Date: Wed, 08 Apr 2026 12:52:00 GMT
• Title: Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training
• Abstract summary: We present a sparse ViT framework for learning reusable representations from heterogeneous detector data.<n>Self-supervised pre-training combines masked autoencoder reconstruction with relational voxel-level objectives.<n>We find that pre-training consistently improves neutrino flavour and charm-quark identification.
• Score: 0.4151704012621244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accelerator-based neutrino physics is entering an energy-frontier regime in which interactions reach the TeV scale and produce exceptionally dense, overlapping detector signatures. In this regime, event interpretation becomes impractical for conventional reconstruction approaches, particularly when labelled data are scarce and the analysis spans diverse downstream objectives. We present a sparse ViT framework for learning reusable representations from heterogeneous detector data. Self-supervised pre-training combines masked autoencoder reconstruction with relational voxel-level objectives for hierarchy, ghost and particle identification, and the resulting shared encoder is then jointly fine-tuned across classification and regression tasks. Evaluated on simulated events from the proposed FASERCal concept at the LHC, we find that pre-training consistently improves neutrino flavour and charm-quark identification, momentum regression, and vertex reconstruction over training from scratch, with the addition of relational objectives yielding further gains in the most topologically complex channels. Interpretability analyses further show that pre-training yields a more structured latent space, while detector-subsystem ablations recover physically plausible channel-dependent roles for the heterogeneous inputs. A data-efficiency study shows that, with roughly $10^3$ labelled events, the pre-trained encoder already matches the flavour-classification performance of a randomly initialised model trained on an order of magnitude more data. The learned representations also transfer effectively to publicly available benchmarks spanning different detector technologies and energy scales, matching or exceeding published baselines. These results support self-supervised pre-training on multimodal detector data as a scalable route towards reusable representations for neutrino and particle-detector analysis.

Related papers

• Robust Quantum Machine Learning for Collider Event Selection under Detector Variability [0.0]
  We investigate whether parameterised quantum models provide a useful inductive bias for robust collider-event selection.<n>In the unsupervised study, quantum autoencoders trained on background events are compared with classical and variational autoencoders for anomaly detection.<n>Under smearing, the quantum models generally exhibit smaller shifts in their output scores and retain their discrimination more effectively than the expressive classical baselines.
  arXiv  Detail & Related papers  (2026-08-11T18:29:08Z)
• Graph-Theoretic Neural Network Fragmentation with Covariant Direct Molecular Force Learning: Enabling Coupled-Cluster Accuracy AIMD for Fluxional Systems [1.476367559174889]
  We present a graph-theoretic molecular fragmentation framework integrated with machine learning to model post-Hartree-Fock nuclear forces.<n>Our framework successfully reproduced complex dynamical signatures and key structural characteristics.<n>Ultimately, this scalable, systematically improvable framework bridges the gap between high-level correlated wavefunction theories and long-timescale reactive sampling.
  arXiv  Detail & Related papers  (2026-07-23T19:49:22Z)
• Human-Machine Collaboration on Generative Meta-Learning: Model and Algorithm [22.212135025785244]
  Generalizing machine learning models to environments that differ from their training distribution remains a critical hurdle.<n>We propose Generative Meta-Learning with Human Feedback (GMHF), a novel framework that bridges this domain gap by leveraging expert intuition to guide data synthesis.
  arXiv  Detail & Related papers  (2026-07-01T13:29:54Z)
• Machine-learned particle flow as a foundation model for collider physics [0.29157078644492396]
  We show that casting event reconstruction as a machine learning problem naturally produces such a shared representation.<n>We demonstrate that a single linear layer trained using only the latent representations achieves competitive performance against state-of-the-art baseline architectures.
  arXiv  Detail & Related papers  (2026-06-12T12:03:04Z)
• K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles [65.21921642886367]
  K-Models is a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles.<n>The proposed method is evaluated through simulations and real-world applications.
  arXiv  Detail & Related papers  (2026-05-14T13:35:44Z)
• Large Vision-Language Models Get Lost in Attention [51.851592109135716]
  We propose a unified framework grounded in information theory and geometry to quantify the geometric and entropic nature of residual updates.<n>Applying this unified framework reveals a fundamental functional decoupling: Attention acts as a subspace-language operator focused on reconfiguration, whereas FFNs serve as subspace-language operators driving semantic innovation.
  arXiv  Detail & Related papers  (2026-05-07T04:45:52Z)
• Benchmarking neural surrogates on realistic spatiotemporal multiphysics flows [18.240532888032394]
  We present REALM (REalistic AI Learning for Multiphysics), a rigorous benchmarking framework designed to test neural surrogates on challenging, application-driven reactive flows.<n>We benchmark over a dozen representative surrogate model families, including spectral operators, convolutional models, Transformers, pointwise operators, and graph/mesh networks.<n>We identify three robust trends: (i) a scaling barrier governed jointly by dimensionality, stiffness, and mesh irregularity, leading to rapidly growing rollout errors; (ii) performance primarily controlled by architectural inductive biases rather than parameter count; and (iii) a persistent gap between nominal accuracy metrics and physically
  arXiv  Detail & Related papers  (2025-12-21T05:04:13Z)
• Quantum-Aware Generative AI for Materials Discovery: A Framework for Robust Exploration Beyond DFT Biases [0.0]
  We introduce a quantum-aware generative AI framework for materials discovery.<n>We implement a robust active learning loop that quantifies and targets the divergence between low- and high-fidelity predictions.<n>Our results demonstrate a 3-5x improvement in successfully identifying potentially stable candidates in high-divergence regions.
  arXiv  Detail & Related papers  (2025-12-13T11:17:21Z)
• Flexible Gravitational-Wave Parameter Estimation with Transformers [73.44614054040267]
  We introduce a flexible transformer-based architecture paired with a training strategy that enables adaptation to diverse analysis settings at inference time.<n>We demonstrate that a single flexible model -- called Dingo-T1 -- can analyze 48 gravitational-wave events from the third LIGO-Virgo-KAGRA Observing Run.
  arXiv  Detail & Related papers  (2025-12-02T17:49:08Z)
• Improving Deepfake Detection with Reinforcement Learning-Based Adaptive Data Augmentation [60.04281435591454]
  CRDA (Curriculum Reinforcement-Learning Data Augmentation) is a novel framework guiding detectors to progressively master multi-domain forgery features.<n>Central to our approach is integrating reinforcement learning and causal inference.<n>Our method significantly improves detector generalizability, outperforming SOTA methods across multiple cross-domain datasets.
  arXiv  Detail & Related papers  (2025-11-10T12:45:52Z)
• Latent Representation Learning in Heavy-Ion Collisions with MaskPoint Transformer [2.6610943214001765]
  We introduce a Transformer-based autoencoder trained with a two-stage paradigm: self-supervised pre-training followed by supervised fine-tuning.<n>The encoder learns latent representations directly from unlabeled HIC data, providing a compact and information-rich feature space.<n>Results establish our two-stage framework as a general and robust foundation for feature learning in HIC, opening the door to more powerful analyses of quark--gluon plasma properties.
  arXiv  Detail & Related papers  (2025-10-08T06:27:10Z)
• Spatial-Temporal-Spectral Unified Modeling for Remote Sensing Dense Prediction [20.1863553357121]
  Current deep learning architectures for remote sensing are fundamentally rigid.<n>We introduce the Spatial-Temporal-Spectral Unified Network (STSUN) for unified modeling.<n> STSUN can adapt to input and output data with arbitrary spatial sizes, temporal lengths, and spectral bands.<n>It unifies various dense prediction tasks and diverse semantic class predictions.
  arXiv  Detail & Related papers  (2025-05-18T07:39:17Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• CycleHOI: Improving Human-Object Interaction Detection with Cycle Consistency of Detection and Generation [37.45945633515955]
  We propose a new learning framework, coined as CycleHOI, to boost the performance of human-object interaction (HOI) detection. Our key design is to introduce a novel cycle consistency loss for the training of HOI detector. We perform extensive experiments to verify the effectiveness and generalization power of our CycleHOI.
  arXiv  Detail & Related papers  (2024-07-16T06:55:43Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Scalable, End-to-End, Deep-Learning-Based Data Reconstruction Chain for Particle Imaging Detectors [0.0]
  This paper introduces an end-to-end, ML-based data reconstruction chain for Liquid Time Projection Chambers (LArTPCs) It is the first implementation to handle the unprecedented pile-up of dozens of high energy neutrino interactions.
  arXiv  Detail & Related papers  (2021-02-01T18:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.