Incremental Structure Discovery of Classification via Sequential Monte Carlo

• URL: http://arxiv.org/abs/2408.07875v1
• Date: Thu, 15 Aug 2024 01:23:49 GMT
• Title: Incremental Structure Discovery of Classification via Sequential Monte Carlo
• Authors: Changze Huang, Di Wang,
• Abstract summary: This paper presents a novel method to automatically discover models of classification on complex data with little prior knowledge. Our method is able to automatically incorporate various features of kernels on synthesized data and real-world data for classification.
• Score: 5.1581069235093295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gaussian Processes (GPs) provide a powerful framework for making predictions and understanding uncertainty for classification with kernels and Bayesian non-parametric learning. Building such models typically requires strong prior knowledge to define preselect kernels, which could be ineffective for online applications of classification that sequentially process data because features of data may shift during the process. To alleviate the requirement of prior knowledge used in GPs and learn new features from data that arrive successively, this paper presents a novel method to automatically discover models of classification on complex data with little prior knowledge. Our method adapts a recently proposed technique for GP-based time-series structure discovery, which integrates GPs and Sequential Monte Carlo (SMC). We extend the technique to handle extra latent variables in GP classification, such that our method can effectively and adaptively learn a-priori unknown structures of classification from continuous input. In addition, our method adapts new batch of data with updated structures of models. Our experiments show that our method is able to automatically incorporate various features of kernels on synthesized data and real-world data for classification. In the experiments of real-world data, our method outperforms various classification methods on both online and offline setting achieving a 10\% accuracy improvement on one benchmark.

Related papers

• A reproducible comparative study of categorical kernels for Gaussian process regression, with new clustering-based nested kernels [3.848846022367752]
  This paper provides a comparative study of all existing categorical kernels on many test cases investigated so far.<n>We also propose new evaluation metrics inspired by the optimization community, which provide quantitative rankings of the methods across several tasks.<n>When the group structure is unknown or when there is no prior knowledge of such a structure, we propose a new clustering-based strategy.
  arXiv  Detail & Related papers  (2025-10-02T09:36:19Z)
• Federated Learning based on Self-Evolving Gaussian Clustering [0.0]
  We present an Evolving Fuzzy System within the context of Federated Learning.<n>Unlike traditional methods, Federated Learning allows models to be trained locally on clients' devices, sharing only the model parameters with a central server instead of the data.
  arXiv  Detail & Related papers  (2025-08-21T09:32:37Z)
• Improving Model Classification by Optimizing the Training Dataset [3.987352341101438]
  Coresets offer a principled approach to data reduction, enabling efficient learning on large datasets.<n>We present a systematic framework for tuning the coreset generation process to enhance downstream classification quality.
  arXiv  Detail & Related papers  (2025-07-22T16:10:11Z)
• Private Training & Data Generation by Clustering Embeddings [74.00687214400021]
  Differential privacy (DP) provides a robust framework for protecting individual data.<n>We introduce a novel principled method for DP synthetic image embedding generation.<n> Empirically, a simple two-layer neural network trained on synthetically generated embeddings achieves state-of-the-art (SOTA) classification accuracy.
  arXiv  Detail & Related papers  (2025-06-20T00:17:14Z)
• FeNeC: Enhancing Continual Learning via Feature Clustering with Neighbor- or Logit-Based Classification [6.720605329045581]
  We introduce FeNeC (Feature Neighborhood) and FeNeC-Log, incorporating its variant based on the log-likelihood function. Our approach generalizes the existing concept by clustering to capture greater intra-class variability. We demonstrate that two FeNeC variants achieve competitive performance in scenarios where task identities are unknown.
  arXiv  Detail & Related papers  (2025-03-18T14:42:38Z)
• UniCell: Universal Cell Nucleus Classification via Prompt Learning [76.11864242047074]
  We propose a universal cell nucleus classification framework (UniCell) It employs a novel prompt learning mechanism to uniformly predict the corresponding categories of pathological images from different dataset domains. In particular, our framework adopts an end-to-end architecture for nuclei detection and classification, and utilizes flexible prediction heads for adapting various datasets.
  arXiv  Detail & Related papers  (2024-02-20T11:50:27Z)
• ProTeCt: Prompt Tuning for Taxonomic Open Set Classification [59.59442518849203]
  Few-shot adaptation methods do not fare well in the taxonomic open set (TOS) setting. We propose a prompt tuning technique that calibrates the hierarchical consistency of model predictions. A new Prompt Tuning for Hierarchical Consistency (ProTeCt) technique is then proposed to calibrate classification across label set granularities.
  arXiv  Detail & Related papers  (2023-06-04T02:55:25Z)
• Automatic learning algorithm selection for classification via convolutional neural networks [0.0]
  The goal of this study is to learn the inherent structure of the data without identifying meta-features. Experiments with simulated datasets show that the proposed approach achieves nearly perfect performance in identifying linear and nonlinear patterns.
  arXiv  Detail & Related papers  (2023-05-16T01:57:01Z)
• Continual Learning For On-Device Environmental Sound Classification [63.81276321857279]
  We propose a simple and efficient continual learning method for on-device environmental sound classification. Our method selects the historical data for the training by measuring the per-sample classification uncertainty.
  arXiv  Detail & Related papers  (2022-07-15T12:13:04Z)
• Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels [6.372625755672473]
  We propose a new method for approximating active learning acquisition strategies that are based on retraining with hypothetically-labeled candidate data points. Although this is usually infeasible with deep networks, we use the neural tangent kernel to approximate the result of retraining.
  arXiv  Detail & Related papers  (2022-06-25T06:13:27Z)
• Incremental Ensemble Gaussian Processes [53.3291389385672]
  We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions. The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
  arXiv  Detail & Related papers  (2021-10-13T15:11:25Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• Few-Shot Incremental Learning with Continually Evolved Classifiers [46.278573301326276]
  Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points. The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems. We propose a Continually Evolved CIF ( CEC) that employs a graph model to propagate context information between classifiers for adaptation.
  arXiv  Detail & Related papers  (2021-04-07T10:54:51Z)
• Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior [26.52863547394537]
  We present a novel probabilistic algorithm to learn a kernel composition by handling the sparsity in the kernel selection with Horseshoe prior. Our model can capture characteristics of time series with significant reductions in computational time and have competitive regression performance on real-world data sets.
  arXiv  Detail & Related papers  (2020-12-21T13:41:15Z)
• An Online Learning Algorithm for a Neuro-Fuzzy Classifier with Mixed-Attribute Data [9.061408029414455]
  General fuzzy min-max neural network (GFMMNN) is one of the efficient neuro-fuzzy systems for data classification. This paper proposes an extended online learning algorithm for the GFMMNN. The proposed method can handle the datasets with both continuous and categorical features.
  arXiv  Detail & Related papers  (2020-09-30T13:45:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.