Nonlinear Multiple Response Regression and Learning of Latent Spaces

• URL: http://arxiv.org/abs/2503.21608v1
• Date: Thu, 27 Mar 2025 15:28:06 GMT
• Title: Nonlinear Multiple Response Regression and Learning of Latent Spaces
• Authors: Ye Tian, Sanyou Wu, Long Feng,
• Abstract summary: We introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings.<n>Unlike other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity.
• Score: 2.6113259186042876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying low-dimensional latent structures within high-dimensional data has long been a central topic in the machine learning community, driven by the need for data compression, storage, transmission, and deeper data understanding. Traditional methods, such as principal component analysis (PCA) and autoencoders (AE), operate in an unsupervised manner, ignoring label information even when it is available. In this work, we introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings. We formulate the problem as a nonlinear multiple-response regression within an index model context. By applying the generalized Stein's lemma, the latent space can be estimated without knowing the nonlinear link functions. Our method can be viewed as a nonlinear generalization of PCA. Moreover, unlike AE and other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity while providing strong theoretical guarantees. Comprehensive numerical experiments and real data analyses demonstrate the superior performance of our method.

Related papers

• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• A Study on Variants of Conventional, Fuzzy, and Nullspace-Based Independence Criteria for Improving Supervised and Unsupervised Learning [0.0]
  We reviewed all independence criteria to design unsupervised learners.<n>We used 3 independence criteria and used them to design unsupervised and supervised dimensionality reduction methods.
  arXiv  Detail & Related papers  (2025-07-22T22:02:50Z)
• A Random Matrix Analysis of In-context Memorization for Nonlinear Attention [18.90197287760915]
  We show that nonlinear Attention incurs higher memorization error than linear ridge regression on random inputs.<n>Our results reveal how nonlinearity and input structure interact with each other to govern the memorization performance of nonlinear Attention.
  arXiv  Detail & Related papers  (2025-06-23T13:56:43Z)
• Interpretable Feature Interaction via Statistical Self-supervised Learning on Tabular Data [22.20955211690874]
  Spofe is a novel self-supervised machine learning pipeline that captures principled representation to achieve clear interpretability with statistical rigor.<n>Underpinning our approach is a robust theoretical framework that delivers precise error bounds and rigorous false discovery rate (FDR) control.<n>Experiments on diverse real-world datasets demonstrate the effectiveness of Spofe.
  arXiv  Detail & Related papers  (2025-03-23T12:27:42Z)
• Adversarial Dependence Minimization [78.36795688238155]
  This work provides a differentiable and scalable algorithm for dependence minimization that goes beyond linear pairwise decorrelation. We demonstrate its utility in three applications: extending PCA to nonlinear decorrelation, improving the generalization of image classification methods, and preventing dimensional collapse in self-supervised representation learning.
  arXiv  Detail & Related papers  (2025-02-05T14:43:40Z)
• Neural Network Approximation for Pessimistic Offline Reinforcement Learning [17.756108291816908]
  We present a non-asymptotic estimation error of pessimistic offline RL using general neural network approximation. Our result shows that the estimation error consists of two parts: the first converges to zero at a desired rate on the sample size with partially controllable concentrability, and the second becomes negligible if the residual constraint is tight.
  arXiv  Detail & Related papers  (2023-12-19T05:17:27Z)
• Identifiable Feature Learning for Spatial Data with Nonlinear ICA [18.480534062833673]
  We introduce a new nonlinear ICA framework that employs latent components which apply naturally to data with higher-dimensional dependency structures. In particular, we develop a new learning and algorithm that extends variational methods to handle the combination of a deep neural network mixing function with the TP prior inducing computational efficacy.
  arXiv  Detail & Related papers  (2023-11-28T15:00:11Z)
• Unsupervised Anomaly Detection via Nonlinear Manifold Learning [0.0]
  Anomalies are samples that significantly deviate from the rest of the data and their detection plays a major role in building machine learning models. We introduce a robust, efficient, and interpretable methodology based on nonlinear manifold learning to detect anomalies in unsupervised settings.
  arXiv  Detail & Related papers  (2023-06-15T18:48:10Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• What Can We Learn from Unlearnable Datasets? [107.12337511216228]
  Unlearnable datasets have the potential to protect data privacy by preventing deep neural networks from generalizing. It is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization. In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured.
  arXiv  Detail & Related papers  (2023-05-30T17:41:35Z)
• Learning Fast Approximations of Sparse Nonlinear Regression [50.00693981886832]
  In this work, we bridge the gap by introducing the Threshold Learned Iterative Shrinkage Algorithming (NLISTA) Experiments on synthetic data corroborate our theoretical results and show our method outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-26T11:31:08Z)
• Nonlinear ISA with Auxiliary Variables for Learning Speech Representations [51.9516685516144]
  We introduce a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables. We propose an algorithm that learns unsupervised speech representations whose subspaces are independent.
  arXiv  Detail & Related papers  (2020-07-25T14:53:09Z)
• Linear Tensor Projection Revealing Nonlinearity [0.294944680995069]
  Dimensionality reduction is an effective method for learning high-dimensional data. We propose a method that searches for a subspace that maximizes the prediction accuracy while retaining as much of the original data information as possible.
  arXiv  Detail & Related papers  (2020-07-08T06:10:39Z)
• Learning Stochastic Behaviour from Aggregate Data [52.012857267317784]
  Learning nonlinear dynamics from aggregate data is a challenging problem because the full trajectory of each individual is not available. We propose a novel method using the weak form of Fokker Planck Equation (FPE) to describe the density evolution of data in a sampled form. In such a sample-based framework we are able to learn the nonlinear dynamics from aggregate data without explicitly solving the partial differential equation (PDE) FPE.
  arXiv  Detail & Related papers  (2020-02-10T03:20:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.