Which Spaces can be Embedded in $L_p$-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

• URL: http://arxiv.org/abs/2410.11116v2
• Date: Wed, 16 Oct 2024 03:28:06 GMT
• Title: Which Spaces can be Embedded in $L_p$-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy
• Authors: Yiping Lu, Daozhe Lin, Qiang Du,
• Abstract summary: We establish a novel connection between the metric entropy growth and the embeddability of function spaces into reproducing kernel Hilbert/Banach spaces. Our results shed new light on the power and limitations of kernel methods for learning complex function spaces.
• Score: 4.256898347232072
• License:
• Abstract: In this paper, we establish a novel connection between the metric entropy growth and the embeddability of function spaces into reproducing kernel Hilbert/Banach spaces. Metric entropy characterizes the information complexity of function spaces and has implications for their approximability and learnability. Classical results show that embedding a function space into a reproducing kernel Hilbert space (RKHS) implies a bound on its metric entropy growth. Surprisingly, we prove a \textbf{converse}: a bound on the metric entropy growth of a function space allows its embedding to a $L_p-$type Reproducing Kernel Banach Space (RKBS). This shows that the ${L}_p-$type RKBS provides a broad modeling framework for learnable function classes with controlled metric entropies. Our results shed new light on the power and limitations of kernel methods for learning complex function spaces.

Related papers

• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Global universal approximation of functional input maps on weighted spaces [3.8059763597999012]
  We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. We prove a global universal approximation result on weighted spaces for continuous functions going beyond the usual approximation on compact sets. We emphasize that the reproducing Hilbert kernel space of the signature kernels are Cameron-Martin spaces of certain Gaussian processes.
  arXiv  Detail & Related papers  (2023-06-05T23:06:32Z)
• Local Intrinsic Dimensional Entropy [29.519376857728325]
  Most entropy measures depend on the spread of the probability distribution over the sample space $mathcalX|$. In this work, we question the role of cardinality and distribution spread in defining entropy measures for continuous spaces. We find that the average value of the local intrinsic dimension of a distribution, denoted as ID-Entropy, can serve as a robust entropy measure for continuous spaces.
  arXiv  Detail & Related papers  (2023-04-05T04:36:07Z)
• Recursive Estimation of Conditional Kernel Mean Embeddings [0.0]
  Kernel mean embeddings map probability distributions to elements of a kernel reproducing Hilbert space (RKHS) We present a new algorithm to estimate the conditional kernel mean map in a Hilbert space valued $L$ space, that is in a Bochner space.
  arXiv  Detail & Related papers  (2023-02-12T16:55:58Z)
• Sobolev Spaces, Kernels and Discrepancies over Hyperspheres [4.521119623956821]
  This work provides theoretical foundations for kernel methods in the hyperspherical context. We characterise the native spaces (reproducing kernel Hilbert spaces) and the Sobolev spaces associated with kernels defined over hyperspheres. Our results have direct consequences for kernel cubature, determining the rate of convergence of the worst case error, and expanding the applicability of cubature algorithms.
  arXiv  Detail & Related papers  (2022-11-16T20:31:38Z)
• Continuous percolation in a Hilbert space for a large system of qubits [58.720142291102135]
  The percolation transition is defined through the appearance of the infinite cluster. We show that the exponentially increasing dimensionality of the Hilbert space makes its covering by finite-size hyperspheres inefficient. Our approach to the percolation transition in compact metric spaces may prove useful for its rigorous treatment in other contexts.
  arXiv  Detail & Related papers  (2022-10-15T13:53:21Z)
• Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings [97.12538243736705]
  We study reinforcement learning with function approximation for large-scale Partially Observable Decision Processes (POMDPs) Our algorithm provably scales to large-scale POMDPs.
  arXiv  Detail & Related papers  (2022-06-24T05:13:35Z)
• Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
  We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces. We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
  arXiv  Detail & Related papers  (2022-02-11T13:51:30Z)
• Value Function Approximations via Kernel Embeddings for No-Regret Reinforcement Learning [10.828727066443909]
  We propose an online model-based RL algorithm, namely the CME-RL, that learns representations of transition distributions as embeddings in a kernel Hilbert space. We demonstrate the efficiency of our algorithm by proving a frequentist (worst-case) regret bound that is of order $tildeObig(Hgamma_NsqrtNbig)$footnote $tildeO(cdot)$ hides only absolute constant and poly-logarithmic factors.
  arXiv  Detail & Related papers  (2020-11-16T11:40:55Z)
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces [208.67848059021915]
  We study the exploration-exploitation tradeoff at the core of reinforcement learning. In particular, we prove that the complexity of the function class $mathcalF$ characterizes the complexity of the function. Our regret bounds are independent of the number of episodes.
  arXiv  Detail & Related papers  (2020-11-09T18:32:22Z)
• The role of feature space in atomistic learning [62.997667081978825]
  Physically-inspired descriptors play a key role in the application of machine-learning techniques to atomistic simulations. We introduce a framework to compare different sets of descriptors, and different ways of transforming them by means of metrics and kernels. We compare representations built in terms of n-body correlations of the atom density, quantitatively assessing the information loss associated with the use of low-order features.
  arXiv  Detail & Related papers  (2020-09-06T14:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.