Rotation-Based Subspace Tracking for Robust Kernel PCA on Streaming Data

• URL: http://arxiv.org/abs/2609.15488v1
• Date: Mon, 14 Sep 2026 12:41:49 GMT
• Title: Rotation-Based Subspace Tracking for Robust Kernel PCA on Streaming Data
• Abstract summary: Real-world data often contains nonlinear patterns, which traditional PCA cannot extract.<n>This paper introduces a rotation-based update mechanism, which updates the subspace estimate by rotating it toward each new incoming feature vector.<n>We show that per-sample rotations converge faster than gradient descent alone, demonstrating an effective mechanism for dynamically tracking a nonlinear subspace in streaming data.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning models process large amounts of data, and Principal Component Analysis (PCA) is a widely used technique to reduce the dimensionality of the data and extract useful features. In practice, datasets often change over time (data drift) and/or arrive one sample at a time (streaming data), making it infeasible to process the entire dataset at once in batch mode. Real-world data also often contains nonlinear patterns, which traditional PCA cannot extract. Kernel PCA addresses this by implicitly mapping samples into a Reproducing Kernel Hilbert Space (RKHS). Raw data also often contains outliers, which can have an outsized effect on the estimated subspace unless the algorithm is made robust. However, existing online robust kernel PCA algorithms are designed to converge to a subspace that is assumed to be fixed, and gradient-descent-based updates lose their effectiveness at tracking further changes once this initial alignment is achieved. This paper introduces a rotation-based update mechanism, which updates the subspace estimate by rotating it toward each new incoming feature vector in Reproducing Kernel Hilbert Space, rather than relying on gradient descent alone. We present two complementary rotation strategies, and show that the extent of rotation can be moderated by a robust influence function to mitigate the effect of outliers. Through experiments on synthetic streaming data with a known ground-truth subspace, we show that per-sample rotations converge faster than gradient descent alone, demonstrating an effective mechanism for dynamically tracking a nonlinear subspace in streaming data.

Related papers

• When to Retrain after Drift: A Data-Only Test of Post-Drift Data Size Sufficiency [17.084943743566217]
  CALIPER is a detector- and model-agnostic, data-only test that estimates the post-drift data size required for stable retraining.<n>We show that CALIPER consistently matches or exceeds the best fixed data size for retraining while incurring negligible overhead and often outperforming incremental updates.
  arXiv  Detail & Related papers  (2026-03-09T23:43:56Z)
• Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space [66.51378598755933]
  We propose a novel method, called Rotation Control Unlearning (RCU), to quantify and control the unlearning degree in the continuous unlearning process.<n>The skew symmetric loss is designed to construct the existence of the cognitive rotation space, where the changes of rotational angle can simulate the continuous unlearning process.<n> Experiments on multiple datasets confirm that our method without retained dataset achieves SOTA performance.
  arXiv  Detail & Related papers  (2025-09-30T03:59:29Z)
• ALPCAH: Subspace Learning for Sample-wise Heteroscedastic Data [15.812312064457867]
  This paper develops a subspace learning method, named ALPCAH, that can estimate the sample-wise noise variances.<n>Our method makes no distributional assumptions of the low-rank component and does not assume that the noise variances are known.<n> Additionally, this paper develops a matrix factorized version of ALPCAH, named LR-ALPCAH, that is much faster and more memory efficient at the cost of requiring subspace dimension to be known or estimated.
  arXiv  Detail & Related papers  (2025-05-12T06:49:47Z)
• DispFormer: A Pretrained Transformer Incorporating Physical Constraints for Dispersion Curve Inversion [56.64622091009756]
  This study introduces DispFormer, a transformer-based neural network for $v_s$ profile inversion from Rayleigh-wave phase and group dispersion curves.<n>DispFormer processes dispersion data independently at each period, allowing it to handle varying lengths without requiring network modifications or strict alignment between training and testing datasets.
  arXiv  Detail & Related papers  (2025-01-08T09:08:24Z)
• Embarassingly Simple Dataset Distillation [0.0]
  We tackle dataset distillation at its core by treating it directly as a bilevel optimization problem. A deeper dive into the nature of distilled data unveils pronounced intercorrelation. We devise a boosting mechanism that generates distilled datasets that contain subsets with near optimal performance across different data budgets.
  arXiv  Detail & Related papers  (2023-11-13T02:14:54Z)
• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• On the Size and Approximation Error of Distilled Sets [57.61696480305911]
  We take a theoretical view on kernel ridge regression based methods of dataset distillation such as Kernel Inducing Points. We prove that a small set of instances exists in the original input space such that its solution in the RFF space coincides with the solution of the original data. A KRR solution can be generated using this distilled set of instances which gives an approximation towards the KRR solution optimized on the full input data.
  arXiv  Detail & Related papers  (2023-05-23T14:37:43Z)
• Capturing the Denoising Effect of PCA via Compression Ratio [3.967854215226183]
  Principal component analysis (PCA) is one of the most fundamental tools in machine learning. In this paper, we propose a novel metric called emphcompression ratio to capture the effect of PCA on high-dimensional noisy data. Building on this new metric, we design a straightforward algorithm that could be used to detect outliers.
  arXiv  Detail & Related papers  (2022-04-22T18:43:47Z)
• Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows [0.0]
  A strategy is proposed to select data points such that they uniformly span the phase-space of the data. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data is available.
  arXiv  Detail & Related papers  (2021-12-28T20:06:28Z)
• SreaMRAK a Streaming Multi-Resolution Adaptive Kernel Algorithm [60.61943386819384]
  Existing implementations of KRR require that all the data is stored in the main memory. We propose StreaMRAK - a streaming version of KRR. We present a showcase study on two synthetic problems and the prediction of the trajectory of a double pendulum.
  arXiv  Detail & Related papers  (2021-08-23T21:03:09Z)
• Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis [65.31074639627226]
  Principal component analysis (PCA) is a technique for extracting the linear structure of a dataset. We propose the deployment of PCA over a multi-access channel based on the algorithm of gradient descent. Over-the-air aggregation is adopted to reduce the multi-access latency, giving the name over-the-air PCA.
  arXiv  Detail & Related papers  (2021-04-20T16:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.