Learning on manifolds without manifold learning
- URL: http://arxiv.org/abs/2402.12687v1
- Date: Tue, 20 Feb 2024 03:27:53 GMT
- Title: Learning on manifolds without manifold learning
- Authors: H. N. Mhaskar and Ryan O'Dowd
- Abstract summary: Function approximation based on data drawn randomly from an unknown distribution is an important problem in machine learning.
One assumes that the data is sampled from an unknown sub-manifold of a high dimensional Euclidean space.
Our approach does not require preprocessing of the data to obtain information about the manifold other than its dimension.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Function approximation based on data drawn randomly from an unknown
distribution is an important problem in machine learning. In contrast to the
prevalent paradigm of solving this problem by minimizing a loss functional, we
have given a direct one-shot construction together with optimal error bounds
under the manifold assumption; i.e., one assumes that the data is sampled from
an unknown sub-manifold of a high dimensional Euclidean space. A great deal of
research deals with obtaining information about this manifold, such as the
eigendecomposition of the Laplace-Beltrami operator or coordinate charts, and
using this information for function approximation. This two step approach
implies some extra errors in the approximation stemming from basic quantities
of the data in addition to the errors inherent in function approximation. In
Neural Networks, 132:253268, 2020, we have proposed a one-shot direct method to
achieve function approximation without requiring the extraction of any
information about the manifold other than its dimension. However, one cannot
pin down the class of approximants used in that paper.
In this paper, we view the unknown manifold as a sub-manifold of an ambient
hypersphere and study the question of constructing a one-shot approximation
using the spherical polynomials based on the hypersphere. Our approach does not
require preprocessing of the data to obtain information about the manifold
other than its dimension. We give optimal rates of approximation for relatively
"rough" functions.
Related papers
- Efficient Prior Calibration From Indirect Data [5.588334720483076]
This paper is concerned with learning the prior model from data, in particular, learning the prior from multiple realizations of indirect data obtained through the noisy observation process.
An efficient residual-based neural operator approximation of the forward model is proposed and it is shown that this may be learned concurrently with the pushforward map.
arXiv Detail & Related papers (2024-05-28T08:34:41Z) - Sketching the Heat Kernel: Using Gaussian Processes to Embed Data [4.220336689294244]
We introduce a novel, non-deterministic method for embedding data in low-dimensional Euclidean space based on realizations of a Gaussian process depending on the geometry of the data.
Our method demonstrates further advantage in its robustness to outliers.
arXiv Detail & Related papers (2024-03-01T22:56:19Z) - Scaling Riemannian Diffusion Models [68.52820280448991]
We show that our method enables us to scale to high dimensional tasks on nontrivial manifold.
We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
arXiv Detail & Related papers (2023-10-30T21:27:53Z) - Implicit Manifold Gaussian Process Regression [49.0787777751317]
Gaussian process regression is widely used to provide well-calibrated uncertainty estimates.
It struggles with high-dimensional data because of the implicit low-dimensional manifold upon which the data actually lies.
In this paper we propose a technique capable of inferring implicit structure directly from data (labeled and unlabeled) in a fully differentiable way.
arXiv Detail & Related papers (2023-10-30T09:52:48Z) - Posterior and Computational Uncertainty in Gaussian Processes [52.26904059556759]
Gaussian processes scale prohibitively with the size of the dataset.
Many approximation methods have been developed, which inevitably introduce approximation error.
This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior.
We develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended.
arXiv Detail & Related papers (2022-05-30T22:16:25Z) - Inferring Manifolds From Noisy Data Using Gaussian Processes [17.166283428199634]
Most existing manifold learning algorithms replace the original data with lower dimensional coordinates.
This article proposes a new methodology for addressing these problems, allowing the estimated manifold between fitted data points.
arXiv Detail & Related papers (2021-10-14T15:50:38Z) - A manifold learning approach for gesture identification from
micro-Doppler radar measurements [1.4610038284393163]
We present a kernel based approximation for manifold learning that does not require the knowledge of anything about the manifold, except its dimension.
We demonstrate the performance of our approach using a publicly available micro-Doppler data set.
arXiv Detail & Related papers (2021-10-04T19:08:44Z) - Manifold Learning via Manifold Deflation [105.7418091051558]
dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data.
Many popular methods can fail dramatically, even on simple two-dimensional Manifolds.
This paper presents an embedding method for a novel, incremental tangent space estimator that incorporates global structure as coordinates.
Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.
arXiv Detail & Related papers (2020-07-07T10:04:28Z) - On the minmax regret for statistical manifolds: the role of curvature [68.8204255655161]
Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models.
We derive a sharper expression than the standard one given by the complexity, where the scalar curvature of the Fisher information metric plays a dominant role.
arXiv Detail & Related papers (2020-07-06T17:28:19Z) - Improved guarantees and a multiple-descent curve for Column Subset
Selection and the Nystr\"om method [76.73096213472897]
We develop techniques which exploit spectral properties of the data matrix to obtain improved approximation guarantees.
Our approach leads to significantly better bounds for datasets with known rates of singular value decay.
We show that both our improved bounds and the multiple-descent curve can be observed on real datasets simply by varying the RBF parameter.
arXiv Detail & Related papers (2020-02-21T00:43:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.