A Tutorial on Regression Analysis: From Linear Models to Deep Learning -- Lecture Notes on Artificial Intelligence

• URL: http://arxiv.org/abs/2512.04747v1
• Date: Thu, 04 Dec 2025 12:35:57 GMT
• Title: A Tutorial on Regression Analysis: From Linear Models to Deep Learning -- Lecture Notes on Artificial Intelligence
• Authors: Jingyuan Wang, Jiahao Ji,
• Abstract summary: This article serves as the regression analysis lecture notes in the Intelligent Computing course cluster.<n>It aims to provide students with a comprehensive and self-contained understanding of regression analysis.
• Score: 14.953701579892302
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This article serves as the regression analysis lecture notes in the Intelligent Computing course cluster (including the courses of Artificial Intelligence, Data Mining, Machine Learning, and Pattern Recognition). It aims to provide students -- who are assumed to possess only basic university-level mathematics (i.e., with prerequisite courses in calculus, linear algebra, and probability theory) -- with a comprehensive and self-contained understanding of regression analysis without requiring any additional references. The lecture notes systematically introduce the fundamental concepts, modeling components, and theoretical foundations of regression analysis, covering linear regression, logistic regression, multinomial logistic regression, polynomial regression, basis-function models, kernel-based methods, and neural-network-based nonlinear regression. Core methodological topics include loss-function design, parameter-estimation principles, ordinary least squares, gradient-based optimization algorithms and their variants, as well as regularization techniques such as Ridge and LASSO regression. Through detailed mathematical derivations, illustrative examples, and intuitive visual explanations, the materials help students understand not only how regression models are constructed and optimized, but also how they reveal the underlying relationships between features and response variables. By bridging classical statistical modeling and modern machine-learning practice, these lecture notes aim to equip students with a solid conceptual and technical foundation for further study in advanced artificial intelligence models.

Related papers

• A Survey on Generative Recommendation: Data, Model, and Tasks [55.36322811257545]
  generative recommendation reconceptualizes recommendation as a generation task rather than discriminative scoring.<n>This survey provides a comprehensive examination through a unified tripartite framework spanning data, model, and task dimensions.<n>We identify five key advantages: world knowledge integration, natural language understanding, reasoning capabilities, scaling laws, and creative generation.
  arXiv  Detail & Related papers  (2025-10-31T04:02:58Z)
• Machine Learning: a Lecture Note [51.31735291774885]
  This lecture note is intended to prepare early-year master's and PhD students in data science or a related discipline with foundational ideas in machine learning.<n>It starts with basic ideas in modern machine learning with classification as a main target task.<n>Based on these basic ideas, the lecture note explores in depth the probablistic approach to unsupervised learning.
  arXiv  Detail & Related papers  (2025-05-06T16:03:41Z)
• Algebraic Machine Learning: Learning as computing an algebraic decomposition of a task [41.94295877935867]
  We propose an alternative foundation based on Abstract Algebra, with mathematics that facilitates the analysis of learning.<n>In this approach, the goal of the task and the data are encoded as axioms of an algebra, and a model is obtained where only these axioms and their logical consequences hold.<n>We validate this new learning principle on standard datasets such as MNIST, FashionMNIST, CIFAR-10, and medical images, achieving performance comparable to optimized multilayer perceptrons.
  arXiv  Detail & Related papers  (2025-02-27T10:13:42Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Tensor Regression [37.35881539885536]
  Regression analysis is a key area of interest in the field of data analysis and machine learning. The emergence of high dimensional data in technologies such as neuroimaging, computer vision, climatology and social networks, has brought challenges to traditional data representation methods. This book provides a systematic study and analysis of vectors-based regression models and their applications in recent years.
  arXiv  Detail & Related papers  (2023-08-22T13:04:12Z)
• Discovering interpretable elastoplasticity models via the neural polynomial method enabled symbolic regressions [0.0]
  Conventional neural network elastoplasticity models are often perceived as lacking interpretability. This paper introduces a two-step machine learning approach that returns mathematical models interpretable by human experts.
  arXiv  Detail & Related papers  (2023-07-24T22:22:32Z)
• Constructing Effective Machine Learning Models for the Sciences: A Multidisciplinary Perspective [77.53142165205281]
  We show how flexible non-linear solutions will not always improve upon manually adding transforms and interactions between variables to linear regression models. We discuss how to recognize this before constructing a data-driven model and how such analysis can help us move to intrinsically interpretable regression models.
  arXiv  Detail & Related papers  (2022-11-21T17:48:44Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)
• Structured learning of rigid-body dynamics: A survey and unified view from a robotics perspective [5.597839822252915]
  We study supervised regression models that combine rigid-body mechanics with data-driven modelling techniques. We provide a unified view on the combination of data-driven regression models, such as neural networks and Gaussian processes, with analytical model priors.
  arXiv  Detail & Related papers  (2020-12-11T11:26:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.