Training toward significance with the decorrelated event classifier transformer neural network

• URL: http://arxiv.org/abs/2401.00428v3
• Date: Thu, 11 Jul 2024 01:50:17 GMT
• Title: Training toward significance with the decorrelated event classifier transformer neural network
• Authors: Jaebak Kim,
• Abstract summary: In natural language processing, one of the leading neural network architectures is the transformer. It is found that this trained network can perform better than boosted decision trees and feed-forward networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Experimental particle physics uses machine learning for many tasks, where one application is to classify signal and background events. This classification can be used to bin an analysis region to enhance the expected significance for a mass resonance search. In natural language processing, one of the leading neural network architectures is the transformer. In this work, an event classifier transformer is proposed to bin an analysis region, in which the network is trained with special techniques. The techniques developed here can enhance the significance and reduce the correlation between the network's output and the reconstructed mass. It is found that this trained network can perform better than boosted decision trees and feed-forward networks.

Related papers

• Opening the Black Box: predicting the trainability of deep neural networks with reconstruction entropy [0.0]
  We present a method for predicting the trainable regime in parameter space for deep feedforward neural networks. For both the MNIST and CIFAR10 datasets, we show that a single epoch of training is sufficient to predict the trainability of the deep feedforward network.
  arXiv  Detail & Related papers  (2024-06-13T18:00:05Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Centered Self-Attention Layers [89.21791761168032]
  The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied. We show that this application inevitably leads to oversmoothing, i.e., to similar representations at the deeper layers. We present a correction term to the aggregating operator of these mechanisms.
  arXiv  Detail & Related papers  (2023-06-02T15:19:08Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Being Friends Instead of Adversaries: Deep Networks Learn from Data Simplified by Other Networks [23.886422706697882]
  A different idea has been recently proposed, named Friendly Training, which consists in altering the input data by adding an automatically estimated perturbation. We revisit and extend this idea inspired by the effectiveness of neural generators in the context of Adversarial Machine Learning. We propose an auxiliary multi-layer network that is responsible of altering the input data to make them easier to be handled by the classifier.
  arXiv  Detail & Related papers  (2021-12-18T16:59:35Z)
• The distance between the weights of the neural network is meaningful [9.329400348695435]
  In the application of neural networks, we need to select a suitable model based on the problem complexity and the dataset scale. This paper proves that the distance between the neural network weights in different training stages can be used to estimate the information accumulated by the network in the training process directly.
  arXiv  Detail & Related papers  (2021-01-31T06:44:49Z)
• Machine Learning Link Inference of Noisy Delay-coupled Networks with Opto-Electronic Experimental Tests [1.0766846340954257]
  We devise a machine learning technique to solve the general problem of inferring network links that have time-delays. We first train a type of machine learning system known as reservoir computing to mimic the dynamics of the unknown network. We formulate and test a technique that uses the trained parameters of the reservoir system output layer to deduce an estimate of the unknown network structure.
  arXiv  Detail & Related papers  (2020-10-29T00:24:13Z)
• Understanding the Role of Individual Units in a Deep Neural Network [85.23117441162772]
  We present an analytic framework to systematically identify hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
  arXiv  Detail & Related papers  (2020-09-10T17:59:10Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Finding trainable sparse networks through Neural Tangent Transfer [16.092248433189816]
  In deep learning, trainable sparse networks that perform well on a specific task are usually constructed using label-dependent pruning criteria. In this article, we introduce Neural Tangent Transfer, a method that instead finds trainable sparse networks in a label-free manner.
  arXiv  Detail & Related papers  (2020-06-15T08:58:01Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.