Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging

• URL: http://arxiv.org/abs/2605.21789v1
• Date: Wed, 20 May 2026 22:31:21 GMT
• Title: Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging
• Abstract summary: Real-time jet tagging is critical for identifying short-lived particle decays in detectors of the Large Hadron Collider.<n>We introduce the Patch Hierarchical Attention Transformer (PHAT-JeT), which combines two mechanisms: a physics-inspired geometric message-passing module that encodes local detector-plane structure, and a hierarchical patch-based attention scheme that computes exact attention within small particle groups.<n>Within a restricted budget, PHAT-JeT achieves state-of-the-art accuracy and background rejection among all resource-constrained jet tagging models on four benchmarks.
• Score: 6.131736935215063
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time jet tagging is critical for identifying short-lived particle decays in the high-throughput detectors of the Large Hadron Collider, where real-time trigger systems responsible for deciding which collision events to store impose strict latency and accuracy constraints. While transformer architectures achieve the highest jet tagging accuracy when compute is unconstrained, their quadratic self-attention cost makes inference restrictive on trigger budget. Existing efficient variants reduce the computational cost, but hinder the classification performance. To address this limitation, we introduce the Patch Hierarchical Attention Transformer (PHAT-JeT), which combines two mechanisms: a physics-inspired geometric message-passing module that encodes local detector-plane structure, and a hierarchical patch-based attention scheme that computes exact attention within small particle groups while preserving global context through lightweight patch-token communication. Within a restricted budget, PHAT-JeT achieves state-of-the-art accuracy and background rejection among all resource-constrained jet tagging models on four benchmarks (\textsc{hls4ml}, JetClass, Top Tagging, and Quark--Gluon). Our code is available at https://github.com/aaronw5/PHAT-JeT.

Related papers

• Scaling Attention via Feature Sparsity [50.64995497733461]
  Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n2 d)$ cost of self-attention.<n>We propose Sparse Feature Attention, where queries and keys are represented as $k$-sparse codes that preserve high-dimensional expressivity.<n>Across GPT-2 and Qwen3 pretraining, SFA matches dense baselines while improving speed by up to $2.5times$ and reducing FLOPs and KV-cache by nearly 50%.
  arXiv  Detail & Related papers  (2026-03-17T08:41:50Z)
• JetFormer: A Scalable and Efficient Transformer for Jet Tagging from Offline Analysis to FPGA Triggers [41.229196563303795]
  We present JetFormer, a versatile and scalable encoder-only Transformer architecture for particle jet tagging at the Large Hadron Collider (LHC)<n>Unlike prior approaches that are often tailored to specific deployment regimes, JetFormer is designed to operate effectively across the full spectrum of jet tagging scenarios.<n>The model processes variable-length sets of particle features without relying on input of explicit pairwise interactions, yet achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-01-23T22:58:25Z)
• PatchBlock: A Lightweight Defense Against Adversarial Patches for Embedded EdgeAI Devices [5.082257334702858]
  Adrial attacks pose a significant challenge to the reliable deployment of machine learning models in EdgeAI applications.<n>We present PatchBlock, a framework designed to detect and neutralize adversarial patches in images.<n>We show that PatchBlock consistently improves robustness, recovering up to 77% of model accuracy under strong patch attacks.
  arXiv  Detail & Related papers  (2026-01-01T15:04:16Z)
• Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging [5.844119137664664]
  Transformers are very effective in capturing both global and local correlations within high-energy particle collisions.<n>Transformers are very effective in capturing both global and local correlations within high-energy particle collisions.<n>We introduce the Spatially Aware Linear Transformer (SAL-T), a physics-inspired enhancement of the linformer architecture that maintains linear attention.
  arXiv  Detail & Related papers  (2025-10-24T18:00:01Z)
• PT$^2$-LLM: Post-Training Ternarization for Large Language Models [52.4629647715623]
  Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment.<n>We propose PT$2$-LLM, a post-training ternarization framework tailored for LLMs.<n>At its core is an Asymmetric Ternary Quantizer equipped with a two-stage refinement pipeline.
  arXiv  Detail & Related papers  (2025-09-27T03:01:48Z)
• JetSeg: Efficient Real-Time Semantic Segmentation Model for Low-Power GPU-Embedded Systems [0.0]
  We propose an efficient model for real-time semantic segmentation called JetSeg. JetSeg consists of an encoder called JetNet, and an improved RegSeg decoder. Our approach outperforms state-of-the-art real-time encoder-decoder models by reducing 46.70M parameters and 5.14% GFLOPs.
  arXiv  Detail & Related papers  (2023-05-19T04:07:26Z)
• Task-Oriented Over-the-Air Computation for Multi-Device Edge AI [57.50247872182593]
  6G networks for supporting edge AI features task-oriented techniques that focus on effective and efficient execution of AI task. Task-oriented over-the-air computation (AirComp) scheme is proposed in this paper for multi-device split-inference system.
  arXiv  Detail & Related papers  (2022-11-02T16:35:14Z)
• Efficient Decoder-free Object Detection with Transformers [75.00499377197475]
  Vision transformers (ViTs) are changing the landscape of object detection approaches. We propose a decoder-free fully transformer-based (DFFT) object detector. DFFT_SMALL achieves high efficiency in both training and inference stages.
  arXiv  Detail & Related papers  (2022-06-14T13:22:19Z)
• Particle Transformer for Jet Tagging [4.604003661048267]
  We present JetClass, a new comprehensive dataset for jet tagging. The dataset consists of 100 M jets, about two orders of magnitude larger than existing public datasets. We propose a new Transformer-based architecture for jet tagging, called Particle Transformer (ParT)
  arXiv  Detail & Related papers  (2022-02-08T10:36:29Z)
• Autoencoders for unsupervised anomaly detection in high energy physics [105.54048699217668]
  We study the tagging of top jet images in a background of QCD jet images. We show that the standard autoencoder setup cannot be considered as a model-independent anomaly tagger. We suggest improved performance measures for the task of model-independent anomaly detection.
  arXiv  Detail & Related papers  (2021-04-19T05:06:57Z)
• End-to-End Object Detection with Transformers [88.06357745922716]
  We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss.
  arXiv  Detail & Related papers  (2020-05-26T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.