jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation

• URL: http://arxiv.org/abs/2601.11719v2
• Date: Wed, 21 Jan 2026 04:25:59 GMT
• Title: jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation
• Authors: Ho Fung Tsoi, Dylan Rankin,
• Abstract summary: jBOT is a pre-training method based on self-distillation for jet data from the CERN Large Hadron Collider.<n>We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space.
• Score: 0.008652091899164643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised learning is a powerful pre-training method for learning feature representations without labels, which often capture generic underlying semantics from the data and can later be fine-tuned for downstream tasks. In this work, we introduce jBOT, a pre-training method based on self-distillation for jet data from the CERN Large Hadron Collider, which combines local particle-level distillation with global jet-level distillation to learn jet representations that support downstream tasks such as anomaly detection and classification. We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space. The clustering in the frozen embedding, when pre-trained on background jets only, enables anomaly detection via simple distance-based metrics, and the learned embedding can be fine-tuned for classification with improved performance compared to supervised models trained from scratch.

Related papers

• ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation [9.230247128710865]
  We propose a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep.<n>ManifoldGD improves representativeness, diversity, and image fidelity without requiring any model retraining.
  arXiv  Detail & Related papers  (2026-02-26T18:07:10Z)
• Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance [51.532841645285835]
  We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution.<n>We show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution.<n>We introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure.
  arXiv  Detail & Related papers  (2026-02-10T19:08:40Z)
• Cutting Through the Noise: On-the-fly Outlier Detection for Robust Training of Machine Learning Interatomic Potentials [0.6999740786886536]
  We introduce an on-the-fly outlier detection scheme that automatically down-weights noisy samples, without requiring additional reference calculations.<n>We show that this approach prevents overfitting and matches the performance of iterative refinement baselines with significantly reduced overhead.<n>We validate its scalability by training a foundation model for organic chemistry on the SPICE dataset, where it reduces energy errors by a factor of three.
  arXiv  Detail & Related papers  (2026-02-09T16:16:22Z)
• CountingDINO: A Training-free Pipeline for Class-Agnostic Counting using Unsupervised Backbones [7.717986156838291]
  Class-agnostic counting (CAC) aims to estimate the number of objects in images without being restricted to predefined categories.<n>Current exemplar-based CAC methods rely heavily on labeled data for training.<n>We introduce CountingDINO, the first training-free exemplar-based CAC framework.
  arXiv  Detail & Related papers  (2025-04-23T09:48:08Z)
• Self-Supervised Pre-Training Boosts Semantic Scene Segmentation on LiDAR Data [0.0]
  We propose to train a self-supervised encoder with Barlow Twins and use it as a pre-trained network in the task of semantic scene segmentation. The experimental results demonstrate that our unsupervised pre-training boosts performance once fine-tuned on the supervised task.
  arXiv  Detail & Related papers  (2023-09-05T11:29:30Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• Semi-Supervised Temporal Action Detection with Proposal-Free Masking [134.26292288193298]
  We propose a novel Semi-supervised Temporal action detection model based on PropOsal-free Temporal mask (SPOT) SPOT outperforms state-of-the-art alternatives, often by a large margin.
  arXiv  Detail & Related papers  (2022-07-14T16:58:47Z)
• Self-supervised Pretraining with Classification Labels for Temporal Activity Detection [54.366236719520565]
  Temporal Activity Detection aims to predict activity classes per frame. Due to the expensive frame-level annotations required for detection, the scale of detection datasets is limited. This work proposes a novel self-supervised pretraining method for detection leveraging classification labels.
  arXiv  Detail & Related papers  (2021-11-26T18:59:28Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.