TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction

• URL: http://arxiv.org/abs/2509.26411v1
• Date: Tue, 30 Sep 2025 15:38:48 GMT
• Title: TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction
• Authors: Sascha Caron, Nadezhda Dobreva, Maarten Kimpel, Uraz Odyurt, Slav Pshenov, Roberto Ruiz de Austri Bazan, Eugene Shalugin, Zef Wolffs, Yue Zhao,
• Abstract summary: High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade.<n>This study extends our work by incorporating loss functions that account for inter-hit correlations, conducting detailed investigations into Transformer attention mechanisms, and a study on the reconstruction of higher-level objects.
• Score: 3.324680086692124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this study, we extend our earlier efforts by incorporating loss functions that account for inter-hit correlations, conducting detailed investigations into (various) Transformer attention mechanisms, and a study on the reconstruction of higher-level objects. Furthermore we discuss new datasets that allow the training on hit level for a range of physics processes. These developments collectively aim to boost both the accuracy, and potentially the efficiency of our tracking models, offering a robust solution to meet the demands of next-generation high-energy physics experiments.

Related papers

• Efficient Training for Human Video Generation with Entropy-Guided Prioritized Progressive Learning [52.25669274734097]
  Ent-Prog is an efficient training framework tailored for diffusion models on human video generation.<n>It reduces both training time and GPU memory consumption while maintaining model performance.
  arXiv  Detail & Related papers  (2025-11-26T07:36:37Z)
• Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders [1.988691274281547]
  We demonstrate transfer learning capabilities in a machine-learned algorithm trained for particle-flow reconstruction in high energy particle colliders.<n>To our knowledge, this is the first full-simulation cross-detector transfer learning study for particle-flow reconstruction.
  arXiv  Detail & Related papers  (2025-02-28T19:16:01Z)
• Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization [88.5582111768376]
  We study the optimization of a Transformer composed of a self-attention layer with softmax followed by a fully connected layer under gradient descent on a certain data distribution model. Our results establish a sharp condition that can distinguish between the small test error phase and the large test error regime, based on the signal-to-noise ratio in the data model.
  arXiv  Detail & Related papers  (2024-09-28T13:24:11Z)
• TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era [2.9052912091435923]
  High-Energy Physics experiments are facing a multi-fold data increase with every new iteration.<n>One such step in need of an overhaul is the task of particle track reconstruction, a.k.a., tracking.<n>A Machine Learning-assisted solution is expected to provide significant improvements.
  arXiv  Detail & Related papers  (2024-07-09T18:47:25Z)
• Novel Approaches for ML-Assisted Particle Track Reconstruction and Hit Clustering [2.7999949281820276]
  Track reconstruction is a vital aspect of High-Energy Physics (HEP) and plays a critical role in major experiments. We utilise a simplified simulator (REDVID) to generate training data that is specifically composed for simplicity. We treat a hit sequence as a hit sequence to track sequence translation problem.
  arXiv  Detail & Related papers  (2024-05-27T16:23:50Z)
• OmniJet-$α$: The first cross-task foundation model for particle physics [0.0]
  Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a variety of downstream applications. We report significant progress on this challenge on several fronts. We demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-$alpha$ model.
  arXiv  Detail & Related papers  (2024-03-08T19:00:01Z)
• Leveraging the Power of Data Augmentation for Transformer-based Tracking [64.46371987827312]
  We propose two data augmentation methods customized for tracking. First, we optimize existing random cropping via a dynamic search radius mechanism and simulation for boundary samples. Second, we propose a token-level feature mixing augmentation strategy, which enables the model against challenges like background interference.
  arXiv  Detail & Related papers  (2023-09-15T09:18:54Z)
• Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors [1.4609888393206634]
  We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. We compare a graph neural network and kernel-based transformer and demonstrate that we can avoid operations while achieving realistic reconstruction. The best graph neural network model shows improvement in the jet transverse momentum resolution by up to 50% compared to the rule-based algorithm.
  arXiv  Detail & Related papers  (2023-09-13T08:16:15Z)
• Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
  We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches. This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
  arXiv  Detail & Related papers  (2023-05-26T00:43:02Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.