Dynamic selection of p-norm in linear adaptive filtering via online kernel-based reinforcement learning

• URL: http://arxiv.org/abs/2210.11317v2
• Date: Fri, 21 Oct 2022 01:19:59 GMT
• Title: Dynamic selection of p-norm in linear adaptive filtering via online kernel-based reinforcement learning
• Authors: Minh Vu, Yuki Akiyama, Konstantinos Slavakis
• Abstract summary: This study addresses the problem of selecting dynamically, at each time instance, the optimal'' p-norm to combat outliers in linear adaptive filtering. Online and data-driven framework is designed via kernel-based reinforcement learning (KBRL)
• Score: 8.319127681936815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study addresses the problem of selecting dynamically, at each time instance, the ``optimal'' p-norm to combat outliers in linear adaptive filtering without any knowledge on the potentially time-varying probability distribution function of the outliers. To this end, an online and data-driven framework is designed via kernel-based reinforcement learning (KBRL). Novel Bellman mappings on reproducing kernel Hilbert spaces (RKHSs) are introduced that need no knowledge on transition probabilities of Markov decision processes, and are nonexpansive with respect to the underlying Hilbertian norm. An approximate policy-iteration framework is finally offered via the introduction of a finite-dimensional affine superset of the fixed-point set of the proposed Bellman mappings. The well-known ``curse of dimensionality'' in RKHSs is addressed by building a basis of vectors via an approximate linear dependency criterion. Numerical tests on synthetic data demonstrate that the proposed framework selects always the ``optimal'' p-norm for the outlier scenario at hand, outperforming at the same time several non-RL and KBRL schemes.

Related papers

• Two-Stage ML-Guided Decision Rules for Sequential Decision Making under Uncertainty [55.06411438416805]
  Sequential Decision Making under Uncertainty (SDMU) is ubiquitous in many domains such as energy, finance, and supply chains. Some SDMU are naturally modeled as Multistage Problems (MSPs) but the resulting optimizations are notoriously challenging from a computational standpoint. This paper introduces a novel approach Two-Stage General Decision Rules (TS-GDR) to generalize the policy space beyond linear functions. The effectiveness of TS-GDR is demonstrated through an instantiation using Deep Recurrent Neural Networks named Two-Stage Deep Decision Rules (TS-LDR)
  arXiv  Detail & Related papers  (2024-05-23T18:19:47Z)
• Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering [3.730504020733928]
  This paper designs novel nonparametric Bellman mappings in reproducing kernel Hilbert spaces (RKHSs) for reinforcement learning (RL) The proposed mappings benefit from the rich approximating properties of RKHSs, adopt no assumptions on the statistics of the data owing to their nonparametric nature, and may operate without any training data. As an application, the proposed mappings are employed to offer a novel solution to the problem of countering outliers in adaptive filtering.
  arXiv  Detail & Related papers  (2024-03-29T07:15:30Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Proximal Bellman mappings for reinforcement learning and their application to robust adaptive filtering [4.140907550856865]
  This paper introduces the novel class of Bellman mappings. The mappings are defined in reproducing kernel Hilbert spaces. An approximate policy-iteration scheme is built on the proposed class of mappings.
  arXiv  Detail & Related papers  (2023-09-14T09:20:21Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Linear Convergence of Reshuffling Kaczmarz Methods With Sparse Constraints [7.936519714074615]
  The Kaczmarz matrix (KZ) and its variants have been extensively studied due to their simplicity and efficiency in solving sub linear equation systems. We provide the first theoretical convergence guarantees for KHT by showing that it converges linearly to the solution of a system with sparsity constraints.
  arXiv  Detail & Related papers  (2023-04-20T07:14:24Z)
• online and lightweight kernel-based approximated policy iteration for dynamic p-norm linear adaptive filtering [8.319127681936815]
  This paper introduces a solution to the problem of selecting dynamically (online) the optimal'' p-norm to combat outliers in linear adaptive filtering. The proposed framework is built on kernel-based reinforcement learning (KBRL)
  arXiv  Detail & Related papers  (2022-10-21T06:29:01Z)
• Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees [5.71097144710995]
  This paper presents a parameterization of nonlinear controllers for uncertain systems building on a recently developed neural network architecture. The proposed framework has "built-in" guarantees of stability, i.e., all policies in the search space result in a contracting (globally exponentially stable) closed-loop system.
  arXiv  Detail & Related papers  (2021-12-02T13:52:37Z)
• Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence [60.20076757208645]
  This paper proposes a general policy mirror descent (GPMD) algorithm for solving regularized RL. We demonstrate that our algorithm converges linearly over an entire range learning rates, in a dimension-free fashion, to the global solution.
  arXiv  Detail & Related papers  (2021-05-24T02:21:34Z)
• On the Stability of Nonlinear Receding Horizon Control: A Geometric Perspective [72.7951562665449]
  widespread adoption of nonlinear Receding Control (RHC) strategies by industry has to more than 30 years. This paper takes the first step towards understanding the role of global geometry in the role of global-based control.
  arXiv  Detail & Related papers  (2021-03-27T22:59:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.