Not All Explanations for Deep Learning Phenomena Are Equally Valuable

• URL: http://arxiv.org/abs/2506.23286v1
• Date: Sun, 29 Jun 2025 15:18:56 GMT
• Title: Not All Explanations for Deep Learning Phenomena Are Equally Valuable
• Authors: Alan Jeffares, Mihaela van der Schaar,
• Abstract summary: We argue that there is little evidence to suggest that counterintuitive phenomena appear in real-world applications.<n>These include double descent, grokking, and the lottery ticket hypothesis.<n>We propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning.
• Score: 58.7010466783654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Developing a better understanding of surprising or counterintuitive phenomena has constituted a significant portion of deep learning research in recent years. These include double descent, grokking, and the lottery ticket hypothesis -- among many others. Works in this area often develop ad hoc hypotheses attempting to explain these observed phenomena on an isolated, case-by-case basis. This position paper asserts that, in many prominent cases, there is little evidence to suggest that these phenomena appear in real-world applications and these efforts may be inefficient in driving progress in the broader field. Consequently, we argue against viewing them as isolated puzzles that require bespoke resolutions or explanations. However, despite this, we suggest that deep learning phenomena do still offer research value by providing unique settings in which we can refine our broad explanatory theories of more general deep learning principles. This position is reinforced by analyzing the research outcomes of several prominent examples of these phenomena from the recent literature. We revisit the current norms in the research community in approaching these problems and propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning.

Related papers

• Recent Deep Learning in Crowd Behaviour Analysis: A Brief Review [9.531376461775462]
  Crowd behaviour analysis is essential to numerous real-world applications, such as public safety and urban planning.<n>The development of deep learning has significantly propelled the research on crowd behaviours.<n>This chapter aims to provide a high-level summary of the ongoing deep learning research in crowd behaviour analysis.
  arXiv  Detail & Related papers  (2025-05-23T22:08:35Z)
• A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond [88.5807076505261]
  Large Reasoning Models (LRMs) have demonstrated strong performance gains by scaling up the length of Chain-of-Thought (CoT) reasoning during inference.<n>A growing concern lies in their tendency to produce excessively long reasoning traces.<n>This inefficiency introduces significant challenges for training, inference, and real-world deployment.
  arXiv  Detail & Related papers  (2025-03-27T15:36:30Z)
• A Comprehensive Survey on Evidential Deep Learning and Its Applications [64.83473301188138]
  Evidential Deep Learning (EDL) provides reliable uncertainty estimation with minimal additional computation in a single forward pass. We first delve into the theoretical foundation of EDL, the subjective logic theory, and discuss its distinctions from other uncertainty estimation frameworks. We elaborate on its extensive applications across various machine learning paradigms and downstream tasks.
  arXiv  Detail & Related papers  (2024-09-07T05:55:06Z)
• Class-wise Activation Unravelling the Engima of Deep Double Descent [0.0]
  Double descent presents a counter-intuitive aspect within the machine learning domain. In this study, we revisited the phenomenon of double descent and discussed the conditions of its occurrence.
  arXiv  Detail & Related papers  (2024-05-13T12:07:48Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [58.107474025048866]
  Forgetting refers to the loss or deterioration of previously acquired knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• Deep Causal Learning: Representation, Discovery and Inference [2.696435860368848]
  Causal learning reveals the essential relationships that underpin phenomena and delineates the mechanisms by which the world evolves. Traditional causal learning methods face numerous challenges and limitations, including high-dimensional variables, unstructured variables, optimization problems, unobserved confounders, selection biases, and estimation inaccuracies. Deep causal learning, which leverages deep neural networks, offers innovative insights and solutions for addressing these challenges.
  arXiv  Detail & Related papers  (2022-11-07T09:00:33Z)
• Theoretical Perspectives on Deep Learning Methods in Inverse Problems [115.93934028666845]
  We focus on generative priors, untrained neural network priors, and unfolding algorithms. In addition to summarizing existing results in these topics, we highlight several ongoing challenges and open problems.
  arXiv  Detail & Related papers  (2022-06-29T02:37:50Z)
• Towards Explaining Adversarial Examples Phenomenon in Artificial Neural Networks [8.31483061185317]
  We study the adversarial examples existence and adversarial training from the standpoint of convergence. We provide evidence that pointwise convergence in ANNs can explain these observations.
  arXiv  Detail & Related papers  (2021-07-22T11:56:14Z)
• In Search of Robust Measures of Generalization [79.75709926309703]
  We develop bounds on generalization error, optimization error, and excess risk. When evaluated empirically, most of these bounds are numerically vacuous. We argue that generalization measures should instead be evaluated within the framework of distributional robustness.
  arXiv  Detail & Related papers  (2020-10-22T17:54:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.