A Random Projection k Nearest Neighbours Ensemble for Classification via Extended Neighbourhood Rule

• URL: http://arxiv.org/abs/2303.12210v1
• Date: Tue, 21 Mar 2023 21:58:59 GMT
• Title: A Random Projection k Nearest Neighbours Ensemble for Classification via Extended Neighbourhood Rule
• Authors: Amjad Ali, Muhammad Hamraz, Dost Muhammad Khan, Wajdan Deebani, Zardad Khan
• Abstract summary: Ensembles based on k nearest neighbours (kNN) combine a large number of base learners. RPExNRule ensemble is proposed where bootstrap samples from the given training data are randomly projected into lower dimensions.
• Score: 0.5052937880533719
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensembles based on k nearest neighbours (kNN) combine a large number of base learners, each constructed on a sample taken from a given training data. Typical kNN based ensembles determine the k closest observations in the training data bounded to a test sample point by a spherical region to predict its class. In this paper, a novel random projection extended neighbourhood rule (RPExNRule) ensemble is proposed where bootstrap samples from the given training data are randomly projected into lower dimensions for additional randomness in the base models and to preserve features information. It uses the extended neighbourhood rule (ExNRule) to fit kNN as base learners on randomly projected bootstrap samples.

Related papers

• Sample size planning for conditional counterfactual mean estimation with a K-armed randomized experiment [0.0]
  We show how to determine a sufficiently large sample size for a $K$-armed randomized experiment. Using policy trees to learn sub-groups, we evaluate our nominal guarantees on a large publicly-available randomized experiment test data set.
  arXiv  Detail & Related papers  (2024-03-06T20:37:29Z)
• Random Boxes Are Open-world Object Detectors [71.86454597677387]
  We show that classifiers trained with random region proposals achieve state-of-the-art Open-world Object Detection (OWOD) We propose RandBox, a Fast R-CNN based architecture trained on random proposals at each training. RandBox significantly outperforms the previous state-of-the-art in all metrics.
  arXiv  Detail & Related papers  (2023-07-17T05:08:32Z)
• MomentDiff: Generative Video Moment Retrieval from Random to Real [71.40038773943638]
  We provide a generative diffusion-based framework called MomentDiff. MomentDiff simulates a typical human retrieval process from random browsing to gradual localization. We show that MomentDiff consistently outperforms state-of-the-art methods on three public benchmarks.
  arXiv  Detail & Related papers  (2023-07-06T09:12:13Z)
• Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network [59.79008107609297]
  We propose in this paper to approximate the joint posterior over the structure of a Bayesian Network. We use a single GFlowNet whose sampling policy follows a two-phase process. Since the parameters are included in the posterior distribution, this leaves more flexibility for the local probability models.
  arXiv  Detail & Related papers  (2023-05-30T19:16:44Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Optimal Extended Neighbourhood Rule $k$ Nearest Neighbours Ensemble [1.8843687952462742]
  A new optimal extended neighborhood rule based ensemble method is proposed in this paper. The ensemble is compared with state-of-the-art methods on 17 benchmark datasets using accuracy, Cohen's kappa, and Brier score (BS)
  arXiv  Detail & Related papers  (2022-11-21T09:13:54Z)
• Intersection of Parallels as an Early Stopping Criterion [64.8387564654474]
  We propose a method to spot an early stopping point in the training iterations without the need for a validation set. For a wide range of learning rates, our method, called Cosine-Distance Criterion (CDC), leads to better generalization on average than all the methods that we compare against.
  arXiv  Detail & Related papers  (2022-08-19T19:42:41Z)
• Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments [5.625056584412003]
  We develop a principled algorithm for estimating the contribution of training data points to a deep learning model. Our algorithm estimates the AME, a quantity that measures the expected (average) marginal effect of adding a data point to a subset of the training data.
  arXiv  Detail & Related papers  (2022-06-20T21:27:18Z)
• A k nearest neighbours classifiers ensemble based on extended neighbourhood rule and features subsets [0.4709844746265484]
  kNN based ensemble methods minimise the effect of outliers by identifying a set of data points in the given feature space that are nearest to an unseen observation. This paper proposes a k nearest neighbour ensemble where the neighbours are determined in k steps.
  arXiv  Detail & Related papers  (2022-05-30T13:57:32Z)
• Distributed Adaptive Nearest Neighbor Classifier: Algorithm and Theory [6.696267547013535]
  We propose a novel distributed adaptive NN classifier for which the number of nearest neighbors is a tuning parameterally chosen by a data-driven criterion. An early stopping rule is proposed when searching for the optimal tuning parameter, which improves the finite sample performance. In particular, we show that when the sub-sample sizes are sufficiently large, the proposed classifier achieves the nearly optimal convergence rate.
  arXiv  Detail & Related papers  (2021-05-20T14:38:41Z)
• Adversarial Examples for $k$-Nearest Neighbor Classifiers Based on Higher-Order Voronoi Diagrams [69.4411417775822]
  Adversarial examples are a widely studied phenomenon in machine learning models. We propose an algorithm for evaluating the adversarial robustness of $k$-nearest neighbor classification.
  arXiv  Detail & Related papers  (2020-11-19T08:49:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.