Fréchet Cumulative Covariance Net for Deep Nonlinear Sufficient Dimension Reduction with Random Objects

• URL: http://arxiv.org/abs/2502.15374v1
• Date: Fri, 21 Feb 2025 10:55:50 GMT
• Title: Fréchet Cumulative Covariance Net for Deep Nonlinear Sufficient Dimension Reduction with Random Objects
• Authors: Hang Yuan, Christina Dan Wang, Zhou Yu,
• Abstract summary: We introduce a new statistical dependence measure termed Fr'echet Cumulative Covariance (FCCov) and develop a novel nonlinear SDR framework based on FCCov.<n>Our approach is not only applicable to complex non-Euclidean data, but also exhibits robustness against outliers.<n>We prove that our method with squared Frobenius norm regularization achieves unbiasedness at the $sigma$-field level.
• Score: 22.156257535146004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nonlinear sufficient dimension reduction\citep{libing_generalSDR}, which constructs nonlinear low-dimensional representations to summarize essential features of high-dimensional data, is an important branch of representation learning. However, most existing methods are not applicable when the response variables are complex non-Euclidean random objects, which are frequently encountered in many recent statistical applications. In this paper, we introduce a new statistical dependence measure termed Fr\'echet Cumulative Covariance (FCCov) and develop a novel nonlinear SDR framework based on FCCov. Our approach is not only applicable to complex non-Euclidean data, but also exhibits robustness against outliers. We further incorporate Feedforward Neural Networks (FNNs) and Convolutional Neural Networks (CNNs) to estimate nonlinear sufficient directions in the sample level. Theoretically, we prove that our method with squared Frobenius norm regularization achieves unbiasedness at the $\sigma$-field level. Furthermore, we establish non-asymptotic convergence rates for our estimators based on FNNs and ResNet-type CNNs, which match the minimax rate of nonparametric regression up to logarithmic factors. Intensive simulation studies verify the performance of our methods in both Euclidean and non-Euclidean settings. We apply our method to facial expression recognition datasets and the results underscore more realistic and broader applicability of our proposal.

Related papers

• Learning a Class of Mixed Linear Regressions: Global Convergence under General Data Conditions [1.9295130374196499]
  Mixed linear regression (MLR) has attracted increasing attention because of its great theoretical and practical importance in nonlinear relationships by utilizing a mixture of linear regression sub-models. Although considerable efforts have been devoted to the learning problem of such systems, most existing investigations impose the strict independent and identically distributed (i.i.d.) or distributed PE conditions.
  arXiv  Detail & Related papers  (2025-03-24T09:57:39Z)
• Deep Fréchet Regression [4.915744683251151]
  We propose a flexible regression model capable of handling high-dimensional predictors without imposing parametric assumptions. The proposed model outperforms existing methods for non-Euclidean responses.
  arXiv  Detail & Related papers  (2024-07-31T07:54:14Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles [0.7499722271664145]
  Neural networks (NNs) often assign high confidence to their predictions, even for points far out-of-distribution. Uncertainty quantification (UQ) is a challenge when they are employed to model interatomic potentials in materials systems. Differentiable UQ techniques can find new informative data and drive active learning loops for robust potentials.
  arXiv  Detail & Related papers  (2023-05-02T19:41:17Z)
• Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets [57.06026574261203]
  We provide theoretical evidence for learning low-dimensional state spaces, which can also model long-term memory. Experiments corroborate our theory, demonstrating extrapolation via learning low-dimensional state spaces with both linear and non-linear RNNs.
  arXiv  Detail & Related papers  (2022-10-25T14:45:15Z)
• Adaptive deep learning for nonlinear time series models [0.0]
  We develop a theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs) We derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models.
  arXiv  Detail & Related papers  (2022-07-06T09:58:58Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Concentration of Non-Isotropic Random Tensors with Applications to Learning and Empirical Risk Minimization [13.572602792770292]
  Dimension is an inherent bottleneck to some modern learning tasks, where optimization methods suffer from the size of the data.<n>We develop tools that aim at reducing these dimensional costs by a dependency on an effective dimension rather than the ambient one.<n>We show the importance of taking advantage of non-isotropic properties in learning problems with the following applications.
  arXiv  Detail & Related papers  (2021-02-04T17:13:03Z)
• Learning Fast Approximations of Sparse Nonlinear Regression [50.00693981886832]
  In this work, we bridge the gap by introducing the Threshold Learned Iterative Shrinkage Algorithming (NLISTA) Experiments on synthetic data corroborate our theoretical results and show our method outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-26T11:31:08Z)
• Deep Dimension Reduction for Supervised Representation Learning [51.10448064423656]
  We propose a deep dimension reduction approach to learning representations with essential characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero.
  arXiv  Detail & Related papers  (2020-06-10T14:47:43Z)
• Nonconvex sparse regularization for deep neural networks and its optimality [1.9798034349981162]
  Deep neural network (DNN) estimators can attain optimal convergence rates for regression and classification problems. We propose a novel penalized estimation method for sparse DNNs. We prove that the sparse-penalized estimator can adaptively attain minimax convergence rates for various nonparametric regression problems.
  arXiv  Detail & Related papers  (2020-03-26T07:15:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.