Neural Networks as Linear Regression: An Introduction for Statisticians

• URL: http://arxiv.org/abs/2606.23601v1
• Date: Mon, 22 Jun 2026 17:04:37 GMT
• Title: Neural Networks as Linear Regression: An Introduction for Statisticians
• Abstract summary: Neural networks are a commonly used prediction tool in computer science and statistics.<n>In this letter, we demystify neural networks by describing networks that approximate a linear regression and describe common customizations that provide a foundation for further study.
• Score: 2.5574009994099196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks are a commonly used prediction tool in computer science and statistics. However, the barrier to entry of this interesting field remains high, particularly for classical statisticians trained in a frequentist perspective. In this letter, we demystify neural networks by describing networks that approximate a linear regression and describe common customizations that provide a foundation for further study.

Related papers

• Predictive Coding Graphs are a Superset of Feedforward Neural Networks [0.0]
  We prove how PCGs define a mathematical superset of feedforward artificial neural networks (multilayer perceptrons)<n>This positions PCNs more strongly within contemporary machine learning (ML)
  arXiv  Detail & Related papers  (2026-03-06T10:50:41Z)
• Hybrid deep additive neural networks [0.0]
  We introduce novel deep neural networks that incorporate the idea of additive regression.<n>Our neural networks share architectural similarities with Kolmogorov-Arnold networks but are based on simpler yet flexible activation and basis functions.<n>We derive their universal approximation properties and demonstrate their effectiveness through simulation studies and a real-data application.
  arXiv  Detail & Related papers  (2024-11-14T04:26:47Z)
• Riemannian Residual Neural Networks [58.925132597945634]
  We show how to extend the residual neural network (ResNet) ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks.
  arXiv  Detail & Related papers  (2023-10-16T02:12:32Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networks [79.28094304325116]
  Despite the capacity of neural nets to learn arbitrary functions, models trained through gradient descent often exhibit a bias towards simpler'' functions. We show how this spectral bias towards low-degree frequencies can in fact hurt the neural network's generalization on real-world datasets. We propose a new scalable functional regularization scheme that aids the neural network to learn higher degree frequencies.
  arXiv  Detail & Related papers  (2023-05-16T20:06:01Z)
• Orthogonal Transforms in Neural Networks Amount to Effective Regularization [0.0]
  We consider applications of neural networks in nonlinear system identification. We show that such a structure is a universal approximator. We empirically show in particular, that such a structure, using the Fourier transform, outperforms equivalent models without orthogonality support.
  arXiv  Detail & Related papers  (2023-05-10T17:52:33Z)
• Neural networks trained with SGD learn distributions of increasing complexity [78.30235086565388]
  We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics. We then exploit higher-order statistics only later during training. We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
  arXiv  Detail & Related papers  (2022-11-21T15:27:22Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Rewiring Networks for Graph Neural Network Training Using Discrete Geometry [0.0]
  Information over-squashing is a problem that significantly impacts the training of graph neural networks (GNNs) In this paper, we investigate the use of discrete analogues of classical geometric notions of curvature to model information flow on networks and rewire them. We show that these classical notions achieve state-of-the-art performance in GNN training accuracy on a variety of real-world network datasets.
  arXiv  Detail & Related papers  (2022-07-16T21:50:39Z)
• Compressive Sensing and Neural Networks from a Statistical Learning Perspective [4.561032960211816]
  We present a generalization error analysis for a class of neural networks suitable for sparse reconstruction from few linear measurements. Under realistic conditions, the generalization error scales only logarithmically in the number of layers, and at most linear in number of measurements.
  arXiv  Detail & Related papers  (2020-10-29T15:05:43Z)
• Statistical Guarantees for Regularized Neural Networks [4.254099382808598]
  We develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer. Our results establish a mathematical basis for regularized estimation of neural networks.
  arXiv  Detail & Related papers  (2020-05-30T15:28:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.