Entropy Loss: An Interpretability Amplifier of 3D Object Detection Network for Intelligent Driving

• URL: http://arxiv.org/abs/2409.00839v1
• Date: Sun, 1 Sep 2024 20:55:50 GMT
• Title: Entropy Loss: An Interpretability Amplifier of 3D Object Detection Network for Intelligent Driving
• Abstract summary: This paper introduces a novel type of loss function, termed "Entropy Loss," along with an innovative training strategy. Entropy Loss is formulated based on the functionality of feature compression networks within the perception model. The accuracy of 3D object detection models using Entropy Loss on the KITTI test set improved by up to 4.47% compared to models without Entropy Loss.
• Score: 12.785134710797244
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: With the increasing complexity of the traffic environment, the significance of safety perception in intelligent driving is intensifying. Traditional methods in the field of intelligent driving perception rely on deep learning, which suffers from limited interpretability, often described as a "black box." This paper introduces a novel type of loss function, termed "Entropy Loss," along with an innovative training strategy. Entropy Loss is formulated based on the functionality of feature compression networks within the perception model. Drawing inspiration from communication systems, the information transmission process in a feature compression network is expected to demonstrate steady changes in information volume and a continuous decrease in information entropy. By modeling network layer outputs as continuous random variables, we construct a probabilistic model that quantifies changes in information volume. Entropy Loss is then derived based on these expectations, guiding the update of network parameters to enhance network interpretability. Our experiments indicate that the Entropy Loss training strategy accelerates the training process. Utilizing the same 60 training epochs, the accuracy of 3D object detection models using Entropy Loss on the KITTI test set improved by up to 4.47\% compared to models without Entropy Loss, underscoring the method's efficacy. The implementation code is available at \url{https://github.com/yhbcode000/Eloss-Interpretability}.

Related papers

• Auto-Compressing Networks [51.221103189527014]
  We introduce Auto-compression Networks (ACNs), an architectural variant where long feedforward connections from each layer replace traditional short residual connections.<n>We show that ACNs exhibit enhanced noise compared to residual networks, superior performance in low-data settings, and mitigate catastrophic forgetting.<n>These findings establish ACNs as a practical approach to developing efficient neural architectures.
  arXiv  Detail & Related papers  (2025-06-11T13:26:09Z)
• A Dynamic Transformer Network for Vehicle Detection [57.4144097001218]
  We present a dynamic Transformer network for vehicle detection (DTNet)<n>DTNet utilizes a dynamic convolution to guide a deep network to dynamically generate weights to enhance adaptability of an obtained detector.<n>To overcome the drawback of difference in an image account, a translation-variant convolution relies on spatial location information to refine obtained structural information for vehicle detection.
  arXiv  Detail & Related papers  (2025-06-03T11:29:35Z)
• Dynamical loss functions shape landscape topography and improve learning in artificial neural networks [0.9208007322096533]
  We show how to transform cross-entropy and mean squared error into dynamical loss functions. We show how they significantly improve validation accuracy for networks of varying sizes.
  arXiv  Detail & Related papers  (2024-10-14T16:27:03Z)
• Causal Context Adjustment Loss for Learned Image Compression [72.7300229848778]
  In recent years, learned image compression (LIC) technologies have surpassed conventional methods notably in terms of rate-distortion (RD) performance. Most present techniques are VAE-based with an autoregressive entropy model, which obviously promotes the RD performance by utilizing the decoded causal context. In this paper, we make the first attempt in investigating the way to explicitly adjust the causal context with our proposed Causal Context Adjustment loss.
  arXiv  Detail & Related papers  (2024-10-07T09:08:32Z)
• Contrastive Representation Learning for Dynamic Link Prediction in Temporal Networks [1.9389881806157312]
  We introduce a self-supervised method for learning representations of temporal networks. We propose a recurrent message-passing neural network architecture for modeling the information flow over time-respecting paths of temporal networks. The proposed method is tested on Enron, COLAB, and Facebook datasets.
  arXiv  Detail & Related papers  (2024-08-22T22:50:46Z)
• Liquid Neural Network-based Adaptive Learning vs. Incremental Learning for Link Load Prediction amid Concept Drift due to Network Failures [37.66676003679306]
  Adapting to concept drift is a challenging task in machine learning. In communication networks, such issue emerges when performing traffic forecasting following afailure event. We propose an approach that exploits adaptive learning algorithms, namely, liquid neural networks, which are capable of self-adaptation to abrupt changes in data patterns without requiring any retraining.
  arXiv  Detail & Related papers  (2024-04-08T08:47:46Z)
• Resilience of Entropy Model in Distributed Neural Networks [7.6340310234573465]
  entropy coding has been introduced to further reduce the communication overhead. In this paper, we investigate resilience of entropy models to intentional interference and unintentional interference. We propose a new defense mechanism that can reduce the transmission overhead of the attacked input by about 9% compared to unperturbed data, with only about 2% accuracy loss.
  arXiv  Detail & Related papers  (2024-03-01T19:39:19Z)
• Disentangling the Causes of Plasticity Loss in Neural Networks [55.23250269007988]
  We show that loss of plasticity can be decomposed into multiple independent mechanisms. We show that a combination of layer normalization and weight decay is highly effective at maintaining plasticity in a variety of synthetic nonstationary learning tasks.
  arXiv  Detail & Related papers  (2024-02-29T00:02:33Z)
• Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy [55.014926694758195]
  Entropy and mutual information in neural networks provide rich information on the learning process. We leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. We show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data.
  arXiv  Detail & Related papers  (2023-12-04T01:32:42Z)
• Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance [0.8749675983608172]
  We derive new mathematical results to measure the changes in entropy as fully-connected and convolutional neural networks process data. By measuring the change in entropy as networks process data effectively, patterns critical to a well-performing network can be visualized and identified. Experiments in image compression, image classification, and image segmentation on benchmark datasets demonstrate these losses guide neural networks to learn rich latent data representations in fewer dimensions.
  arXiv  Detail & Related papers  (2023-08-28T23:33:07Z)
• Effect of Choosing Loss Function when Using T-batching for Representation Learning on Dynamic Networks [0.0]
  T-batching is a valuable technique for training dynamic network models. We have identified a limitation in the training loss function used with t-batching. We propose two alternative loss functions that overcome these issues, resulting in enhanced training performance.
  arXiv  Detail & Related papers  (2023-08-13T23:34:36Z)
• Eloss in the way: A Sensitive Input Quality Metrics for Intelligent Driving [8.125523207635128]
  This paper proposes a new type of metric known as Eloss. Eloss is designed based on an explanation of the perception model's information compression layers. Our experiments demonstrate that Eloss can deviate from the standard value by a factor over 100 with anomalous data.
  arXiv  Detail & Related papers  (2023-02-02T10:11:08Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• Learning Context-Based Non-local Entropy Modeling for Image Compression [140.64888994506313]
  In this paper, we propose a non-local operation for context modeling by employing the global similarity within the context. The entropy model is further adopted as the rate loss in a joint rate-distortion optimization. Considering that the width of the transforms is essential in training low distortion models, we finally produce a U-Net block in the transforms to increase the width with manageable memory consumption and time complexity.
  arXiv  Detail & Related papers  (2020-05-10T13:28:18Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.