Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions

• URL: http://arxiv.org/abs/2307.02478v2
• Date: Sat, 22 Jul 2023 04:33:51 GMT
• Title: Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions
• Authors: Liangchen Liu, Juncai He and Richard Tsai
• Abstract summary: We study linear regression applied to data structured on a manifold. We analyze the impact of the manifold's curvatures on the uniqueness of the regression solution.
• Score: 4.8234611688915665
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold's extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold's curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the embedded submanifold is flat in some dimensions. Otherwise, the manifold's curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.

Related papers

• Pulling back symmetric Riemannian geometry for data analysis [0.0]
  Ideal data analysis tools for data sets should account for non-linear geometry. A rich mathematical structure to account for non-linear geometries has been shown to be able to capture the data geometry. Many standard data analysis tools initially developed for data in Euclidean space can be generalised efficiently to data on a symmetric Riemannian manifold.
  arXiv  Detail & Related papers  (2024-03-11T10:59:55Z)
• Implicit Manifold Gaussian Process Regression [49.0787777751317]
  Gaussian process regression is widely used to provide well-calibrated uncertainty estimates. It struggles with high-dimensional data because of the implicit low-dimensional manifold upon which the data actually lies. In this paper we propose a technique capable of inferring implicit structure directly from data (labeled and unlabeled) in a fully differentiable way.
  arXiv  Detail & Related papers  (2023-10-30T09:52:48Z)
• Conformal inference for regression on Riemannian Manifolds [49.7719149179179]
  We investigate prediction sets for regression scenarios when the response variable, denoted by $Y$, resides in a manifold, and the covariable, denoted by X, lies in Euclidean space. We prove the almost sure convergence of the empirical version of these regions on the manifold to their population counterparts.
  arXiv  Detail & Related papers  (2023-10-12T10:56:25Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Nonlinear Causal Discovery via Kernel Anchor Regression [12.672917592158269]
  We tackle the nonlinear setting by proposing kernel anchor regression (KAR) We provide convergence results for the proposed KAR estimators and the identifiability conditions for KAR to learn the nonlinear structural equation models (SEM) Experimental results demonstrate the superior performances of the proposed KAR estimators over existing baselines.
  arXiv  Detail & Related papers  (2022-10-30T08:46:36Z)
• Shape And Structure Preserving Differential Privacy [70.08490462870144]
  We show how the gradient of the squared distance function offers better control over sensitivity than the Laplace mechanism. We also show how using the gradient of the squared distance function offers better control over sensitivity than the Laplace mechanism.
  arXiv  Detail & Related papers  (2022-09-21T18:14:38Z)
• Inferring Manifolds From Noisy Data Using Gaussian Processes [17.166283428199634]
  Most existing manifold learning algorithms replace the original data with lower dimensional coordinates. This article proposes a new methodology for addressing these problems, allowing the estimated manifold between fitted data points.
  arXiv  Detail & Related papers  (2021-10-14T15:50:38Z)
• A Hypergradient Approach to Robust Regression without Correspondence [85.49775273716503]
  We consider a variant of regression problem, where the correspondence between input and output data is not available. Most existing methods are only applicable when the sample size is small. We propose a new computational framework -- ROBOT -- for the shuffled regression problem.
  arXiv  Detail & Related papers  (2020-11-30T21:47:38Z)
• On the minmax regret for statistical manifolds: the role of curvature [68.8204255655161]
  Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models. We derive a sharper expression than the standard one given by the complexity, where the scalar curvature of the Fisher information metric plays a dominant role.
  arXiv  Detail & Related papers  (2020-07-06T17:28:19Z)
• Sample complexity and effective dimension for regression on manifolds [13.774258153124205]
  We consider the theory of regression on a manifold using kernel reproducing Hilbert space methods. We show that certain spaces of smooth functions on a manifold are effectively finite-dimensional, with a complexity that scales according to the manifold dimension.
  arXiv  Detail & Related papers  (2020-06-13T14:09:55Z)
• Spherical Principal Curves [16.095213132052987]
  We propose a new approach to construct principal curves on a sphere by a projection of the data onto a continuous curve. Our approach lies in the same line of Hastie and Stuetzle (1989) that proposed principal curves for Euclidean space data.
  arXiv  Detail & Related papers  (2020-03-05T12:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.