Kernel Smoothing, Mean Shift, and Their Learning Theory with Directional Data

• URL: http://arxiv.org/abs/2010.13523v2
• Date: Mon, 7 Jun 2021 08:38:34 GMT
• Title: Kernel Smoothing, Mean Shift, and Their Learning Theory with Directional Data
• Authors: Yikun Zhang, Yen-Chi Chen
• Abstract summary: This paper studies both statistical and computational problems of kernel smoothing for directional data. We generalize the classical mean shift algorithm to directional data, which allows us to identify local modes of the directional kernel density estimator (KDE)
• Score: 2.8935588665357077
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Directional data consist of observations distributed on a (hyper)sphere, and appear in many applied fields, such as astronomy, ecology, and environmental science. This paper studies both statistical and computational problems of kernel smoothing for directional data. We generalize the classical mean shift algorithm to directional data, which allows us to identify local modes of the directional kernel density estimator (KDE). The statistical convergence rates of the directional KDE and its derivatives are derived, and the problem of mode estimation is examined. We also prove the ascending property of the directional mean shift algorithm and investigate a general problem of gradient ascent on the unit hypersphere. To demonstrate the applicability of the algorithm, we evaluate it as a mode clustering method on both simulated and real-world data sets.

Related papers

• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Score-based Diffusion Models in Function Space [140.792362459734]
  Diffusion models have recently emerged as a powerful framework for generative modeling. We introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Rigorous dynamical mean field theory for stochastic gradient descent methods [17.90683687731009]
  We prove closed-form equations for the exact high-dimensionals of a family of first order gradient-based methods. This includes widely used algorithms such as gradient descent (SGD) or Nesterov acceleration.
  arXiv  Detail & Related papers  (2022-10-12T21:10:55Z)
• Nonparametric learning of kernels in nonlocal operators [6.314604944530131]
  We provide a rigorous identifiability analysis and convergence study for the learning of kernels in nonlocal operators. We propose a nonparametric regression algorithm with a novel data adaptive RKHS Tikhonov regularization method based on the function space of identifiability.
  arXiv  Detail & Related papers  (2022-05-23T02:47:55Z)
• Data-heterogeneity-aware Mixing for Decentralized Learning [63.83913592085953]
  We characterize the dependence of convergence on the relationship between the mixing weights of the graph and the data heterogeneity across nodes. We propose a metric that quantifies the ability of a graph to mix the current gradients. Motivated by our analysis, we propose an approach that periodically and efficiently optimize the metric.
  arXiv  Detail & Related papers  (2022-04-13T15:54:35Z)
• Linearization and Identification of Multiple-Attractors Dynamical System through Laplacian Eigenmaps [8.161497377142584]
  We propose a Graph-based spectral clustering method that takes advantage of a velocity-augmented kernel to connect data-points belonging to the same dynamics. We prove that there always exist a set of 2-dimensional embedding spaces in which the sub-dynamics are linear, and n-dimensional embedding where they are quasi-linear. We learn a diffeomorphism from the Laplacian embedding space to the original space and show that the Laplacian embedding leads to good reconstruction accuracy and a faster training time.
  arXiv  Detail & Related papers  (2022-02-18T12:43:25Z)
• Partial Counterfactual Identification from Observational and Experimental Data [83.798237968683]
  We develop effective Monte Carlo algorithms to approximate the optimal bounds from an arbitrary combination of observational and experimental data. Our algorithms are validated extensively on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-10-12T02:21:30Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• Space Partitioning and Regression Mode Seeking via a Mean-Shift-Inspired Algorithm [5.990174495635326]
  Mean shift (MS) algorithm is a nonparametric method used to cluster sample points and find the local modes of kernel density estimates. We develop an algorithm to estimate the modes of regression functions and partition the sample points in the input space.
  arXiv  Detail & Related papers  (2021-04-20T16:35:17Z)
• Data Assimilation Networks [1.5545257664210517]
  Data assimilation aims at forecasting the state of a dynamical system by combining a mathematical representation of the system with noisy observations. We propose a fully data driven deep learning architecture generalizing recurrent Elman networks and data assimilation algorithms. Our architecture achieves comparable performance to EnKF on both the analysis and the propagation of probability density functions of the system state at a given time without using any explicit regularization technique.
  arXiv  Detail & Related papers  (2020-10-19T17:35:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.