Multivariate Smoothing via the Fourier Integral Theorem and Fourier Kernel

• URL: http://arxiv.org/abs/2012.14482v1
• Date: Mon, 28 Dec 2020 20:59:42 GMT
• Title: Multivariate Smoothing via the Fourier Integral Theorem and Fourier Kernel
• Authors: Nhat Ho and Stephen G. Walker
• Abstract summary: Rates of convergence are established and, in many cases, provide superior rates to current standard estimators. Rates of convergence are established and, in many cases, provide superior rates to current standard estimators.
• Score: 9.619814126465206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Starting with the Fourier integral theorem, we present natural Monte Carlo estimators of multivariate functions including densities, mixing densities, transition densities, regression functions, and the search for modes of multivariate density functions (modal regression). Rates of convergence are established and, in many cases, provide superior rates to current standard estimators such as those based on kernels, including kernel density estimators and kernel regression functions. Numerical illustrations are presented.

Related papers

• New random projections for isotropic kernels using stable spectral distributions [0.0]
  We decompose spectral kernel distributions as a scale mixture of $alpha$-stable random vectors. Results have broad applications for support vector machines, kernel ridge regression, and other kernel-based machine learning techniques.
  arXiv  Detail & Related papers  (2024-11-05T03:28:01Z)
• Generalizing Stochastic Smoothing for Differentiation and Gradient Estimation [59.86921150579892]
  We deal with the problem of gradient estimation for differentiable relaxations of algorithms, operators, simulators, and other non-differentiable functions. We develop variance reduction strategies for differentiable sorting and ranking, differentiable shortest-paths on graphs, differentiable rendering for pose estimation, as well as differentiable cryo-ET simulations.
  arXiv  Detail & Related papers  (2024-10-10T17:10:00Z)
• Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spaces [52.424621227687894]
  We introduce a unified framework for group equivariant networks on homogeneous spaces. We take advantage of the sparsity of Fourier coefficients of the lifted feature fields. We show that other methods treating features as the Fourier coefficients in the stabilizer subgroup are special cases of our activation.
  arXiv  Detail & Related papers  (2022-06-16T17:59:01Z)
• Transformer with Fourier Integral Attentions [18.031977028559282]
  We propose a new class of transformers in which the dot-product kernels are replaced by the novel generalized Fourier integral kernels. Compared to the conventional transformers with dot-product attention, FourierFormers attain better accuracy and reduce the redundancy between attention heads. We empirically corroborate the advantages of FourierFormers over the baseline transformers in a variety of practical applications including language modeling and image classification.
  arXiv  Detail & Related papers  (2022-06-01T03:06:21Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• On Integral Theorems: Monte Carlo Estimators and Optimal Functions [9.619814126465206]
  We introduce a class of integral theorems based on cyclic functions. The integral theorems provide natural estimators of density functions via Monte Carlo integration. Our proof techniques rely on a variational approach in ordinary differential equations and the Cauchy residue theorem in complex analysis.
  arXiv  Detail & Related papers  (2021-07-22T22:25:21Z)
• Statistical Analysis from the Fourier Integral Theorem [9.619814126465206]
  We look at Monte Carlo based estimators of conditional distribution functions. We study a number of problems, such as prediction for Markov processes. Estimators are explicit Monte Carlo based and require no iterative algorithms.
  arXiv  Detail & Related papers  (2021-06-11T20:44:54Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• The Minecraft Kernel: Modelling correlated Gaussian Processes in the Fourier domain [3.6526103325150383]
  We present a family of kernel that can approximate any stationary multi-output kernel to arbitrary precision. The proposed family of kernel represents the first multi-output generalisation of the spectral mixture kernel.
  arXiv  Detail & Related papers  (2021-03-11T20:54:51Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.