Nonlinear Multiple Response Regression and Learning of Latent Spaces

• URL: http://arxiv.org/abs/2503.21608v1
• Date: Thu, 27 Mar 2025 15:28:06 GMT
• Title: Nonlinear Multiple Response Regression and Learning of Latent Spaces
• Authors: Ye Tian, Sanyou Wu, Long Feng,
• Abstract summary: We introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings.<n>Unlike other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity.
• Score: 2.6113259186042876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying low-dimensional latent structures within high-dimensional data has long been a central topic in the machine learning community, driven by the need for data compression, storage, transmission, and deeper data understanding. Traditional methods, such as principal component analysis (PCA) and autoencoders (AE), operate in an unsupervised manner, ignoring label information even when it is available. In this work, we introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings. We formulate the problem as a nonlinear multiple-response regression within an index model context. By applying the generalized Stein's lemma, the latent space can be estimated without knowing the nonlinear link functions. Our method can be viewed as a nonlinear generalization of PCA. Moreover, unlike AE and other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity while providing strong theoretical guarantees. Comprehensive numerical experiments and real data analyses demonstrate the superior performance of our method.

Related papers

• Interpretable Feature Interaction via Statistical Self-supervised Learning on Tabular Data [22.20955211690874]
  Spofe is a novel self-supervised machine learning pipeline that captures principled representation to achieve clear interpretability with statistical rigor.<n>Underpinning our approach is a robust theoretical framework that delivers precise error bounds and rigorous false discovery rate (FDR) control.<n>Experiments on diverse real-world datasets demonstrate the effectiveness of Spofe.
  arXiv  Detail & Related papers  (2025-03-23T12:27:42Z)
• Adversarial Dependence Minimization [78.36795688238155]
  This work provides a differentiable and scalable algorithm for dependence minimization that goes beyond linear pairwise decorrelation. We demonstrate its utility in three applications: extending PCA to nonlinear decorrelation, improving the generalization of image classification methods, and preventing dimensional collapse in self-supervised representation learning.
  arXiv  Detail & Related papers  (2025-02-05T14:43:40Z)
• Neural Network Approximation for Pessimistic Offline Reinforcement Learning [17.756108291816908]
  We present a non-asymptotic estimation error of pessimistic offline RL using general neural network approximation. Our result shows that the estimation error consists of two parts: the first converges to zero at a desired rate on the sample size with partially controllable concentrability, and the second becomes negligible if the residual constraint is tight.
  arXiv  Detail & Related papers  (2023-12-19T05:17:27Z)
• Identifiable Feature Learning for Spatial Data with Nonlinear ICA [18.480534062833673]
  We introduce a new nonlinear ICA framework that employs latent components which apply naturally to data with higher-dimensional dependency structures. In particular, we develop a new learning and algorithm that extends variational methods to handle the combination of a deep neural network mixing function with the TP prior inducing computational efficacy.
  arXiv  Detail & Related papers  (2023-11-28T15:00:11Z)
• Unsupervised Anomaly Detection via Nonlinear Manifold Learning [0.0]
  Anomalies are samples that significantly deviate from the rest of the data and their detection plays a major role in building machine learning models. We introduce a robust, efficient, and interpretable methodology based on nonlinear manifold learning to detect anomalies in unsupervised settings.
  arXiv  Detail & Related papers  (2023-06-15T18:48:10Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• What Can We Learn from Unlearnable Datasets? [107.12337511216228]
  Unlearnable datasets have the potential to protect data privacy by preventing deep neural networks from generalizing. It is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization. In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured.
  arXiv  Detail & Related papers  (2023-05-30T17:41:35Z)
• Learning Fast Approximations of Sparse Nonlinear Regression [50.00693981886832]
  In this work, we bridge the gap by introducing the Threshold Learned Iterative Shrinkage Algorithming (NLISTA) Experiments on synthetic data corroborate our theoretical results and show our method outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-26T11:31:08Z)
• Nonlinear ISA with Auxiliary Variables for Learning Speech Representations [51.9516685516144]
  We introduce a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables. We propose an algorithm that learns unsupervised speech representations whose subspaces are independent.
  arXiv  Detail & Related papers  (2020-07-25T14:53:09Z)
• Linear Tensor Projection Revealing Nonlinearity [0.294944680995069]
  Dimensionality reduction is an effective method for learning high-dimensional data. We propose a method that searches for a subspace that maximizes the prediction accuracy while retaining as much of the original data information as possible.
  arXiv  Detail & Related papers  (2020-07-08T06:10:39Z)
• Learning Stochastic Behaviour from Aggregate Data [52.012857267317784]
  Learning nonlinear dynamics from aggregate data is a challenging problem because the full trajectory of each individual is not available. We propose a novel method using the weak form of Fokker Planck Equation (FPE) to describe the density evolution of data in a sampled form. In such a sample-based framework we are able to learn the nonlinear dynamics from aggregate data without explicitly solving the partial differential equation (PDE) FPE.
  arXiv  Detail & Related papers  (2020-02-10T03:20:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.