CI-RKM: A Class-Informed Approach to Robust Restricted Kernel Machines

• URL: http://arxiv.org/abs/2504.11476v1
• Date: Sat, 12 Apr 2025 11:12:30 GMT
• Title: CI-RKM: A Class-Informed Approach to Robust Restricted Kernel Machines
• Authors: Ritik Mishra, Mushir Akhtar, M. Tanveer,
• Abstract summary: Restricted kernel machines (RKMs) represent a versatile and powerful framework within the kernel machine family.<n>We propose a novel enhancement to the RKM framework by integrating a class-informed weighted function.<n>Our proposed method establishes a significant advancement in the development of kernel-based learning models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Restricted kernel machines (RKMs) represent a versatile and powerful framework within the kernel machine family, leveraging conjugate feature duality to address a wide range of machine learning tasks, including classification, regression, and feature learning. However, their performance can degrade significantly in the presence of noise and outliers, which compromises robustness and predictive accuracy. In this paper, we propose a novel enhancement to the RKM framework by integrating a class-informed weighted function. This weighting mechanism dynamically adjusts the contribution of individual training points based on their proximity to class centers and class-specific characteristics, thereby mitigating the adverse effects of noisy and outlier data. By incorporating weighted conjugate feature duality and leveraging the Schur complement theorem, we introduce the class-informed restricted kernel machine (CI-RKM), a robust extension of the RKM designed to improve generalization and resilience to data imperfections. Experimental evaluations on benchmark datasets demonstrate that the proposed CI-RKM consistently outperforms existing baselines, achieving superior classification accuracy and enhanced robustness against noise and outliers. Our proposed method establishes a significant advancement in the development of kernel-based learning models, addressing a core challenge in the field.

Related papers

• Semi-supervised Semantic Segmentation with Multi-Constraint Consistency Learning [81.02648336552421]
  We propose a Multi-Constraint Consistency Learning approach to facilitate the staged enhancement of the encoder and decoder.<n>Self-adaptive feature masking and noise injection are designed in an instance-specific manner to perturb the features for robust learning of the decoder.<n> Experimental results on Pascal VOC2012 and Cityscapes datasets demonstrate that our proposed MCCL achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-23T03:21:33Z)
• TRKM: Twin Restricted Kernel Machines for Classification and Regression [0.0]
  TRKM combines the benefits of twin models with the robustness of the RKM framework to enhance classification and regression tasks.<n>We implement the TRKM model on the brain age dataset, demonstrating its efficacy in predicting brain age.
  arXiv  Detail & Related papers  (2025-02-13T05:13:46Z)
• One Class Restricted Kernel Machines [0.0]
  Restricted kernel machines (RKMs) have demonstrated a significant impact in enhancing generalization ability in the field of machine learning.<n>RKMs's efficacy can be compromised by the presence of outliers and other forms of contamination within the dataset.<n>To address this critical issue and robustness of the model, we propose the novel one-class RKM (OCRKM)<n>In the framework of OCRKM, we employ an energy function akin to that of the RBM, which integrates both visible and hidden variables in a nonprobabilistic setting.
  arXiv  Detail & Related papers  (2025-02-11T07:11:20Z)
• Enhancing Robust Fairness via Confusional Spectral Regularization [6.041034366572273]
  We derive a robust generalization bound for the worst-class robust error within the PAC-Bayesian framework.<n>We propose a novel regularization technique to improve worst-class robust accuracy and enhance robust fairness.
  arXiv  Detail & Related papers  (2025-01-22T23:32:19Z)
• Nonparametric Sparse Online Learning of the Koopman Operator [11.710740395697128]
  The Koopman operator provides a powerful framework for representing the dynamics of general nonlinear dynamical systems.<n>Data-driven techniques to learn the Koopman operator typically assume that the chosen function space is closed under system dynamics.<n>We present an operator approximation algorithm to learn the Koopman operator iteratively with control over the complexity of the representation.
  arXiv  Detail & Related papers  (2024-05-13T02:18:49Z)
• InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling [66.3072381478251]
  Reward hacking, also termed reward overoptimization, remains a critical challenge. We propose a framework for reward modeling, namely InfoRM, by introducing a variational information bottleneck objective. We show that InfoRM's overoptimization detection mechanism is not only effective but also robust across a broad range of datasets.
  arXiv  Detail & Related papers  (2024-02-14T17:49:07Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• ARBEx: Attentive Feature Extraction with Reliability Balancing for Robust Facial Expression Learning [5.648318448953635]
  ARBEx is a novel attentive feature extraction framework driven by Vision Transformer. We employ learnable anchor points in the embedding space with label distributions and multi-head self-attention mechanism to optimize performance against weak predictions. Our strategy outperforms current state-of-the-art methodologies, according to extensive experiments conducted in a variety of contexts.
  arXiv  Detail & Related papers  (2023-05-02T15:10:01Z)
• Open-Set Recognition: A Good Closed-Set Classifier is All You Need [146.6814176602689]
  We show that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We use this correlation to boost the performance of the cross-entropy OSR 'baseline' by improving its closed-set accuracy. We also construct new benchmarks which better respect the task of detecting semantic novelty.
  arXiv  Detail & Related papers  (2021-10-12T17:58:59Z)
• Learning to Learn Kernels with Variational Random Features [118.09565227041844]
  We introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We formulate the optimization of MetaVRF as a variational inference problem. We show that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.
  arXiv  Detail & Related papers  (2020-06-11T18:05:29Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)
• Robust Generative Restricted Kernel Machines using Weighted Conjugate Feature Duality [11.68800227521015]
  We introduce weighted conjugate feature duality in the framework of Restricted Kernel Machines (RKMs) The RKM formulation allows for an easy integration of methods from classical robust statistics. Experiments show that the weighted RKM is capable of generating clean images when contamination is present in the training data.
  arXiv  Detail & Related papers  (2020-02-04T09:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.