Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification

• URL: http://arxiv.org/abs/2008.10165v1
• Date: Mon, 24 Aug 2020 02:35:50 GMT
• Title: Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification
• Authors: Chuan-Xian Ren, Pengfei Ge, Dao-Qing Dai, Hong Yan
• Abstract summary: A new kernel learning method is proposed to improve the discrimination performance of Conditional Maximum Mean Discrepancy (CMMD) It can be operated with deep network features iteratively and thus denoted as KLN for abbreviation. In particular, the kernel-based similarities are iteratively learned on the deep network features, and the algorithm can be implemented in an end-to-end manner.
• Score: 26.09932710494144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional Maximum Mean Discrepancy (CMMD) can capture the discrepancy between conditional distributions by drawing support from nonlinear kernel functions, thus it has been successfully used for pattern classification. However, CMMD does not work well on complex distributions, especially when the kernel function fails to correctly characterize the difference between intra-class similarity and inter-class similarity. In this paper, a new kernel learning method is proposed to improve the discrimination performance of CMMD. It can be operated with deep network features iteratively and thus denoted as KLN for abbreviation. The CMMD loss and an auto-encoder (AE) are used to learn an injective function. By considering the compound kernel, i.e., the injective function with a characteristic kernel, the effectiveness of CMMD for data category description is enhanced. KLN can simultaneously learn a more expressive kernel and label prediction distribution, thus, it can be used to improve the classification performance in both supervised and semi-supervised learning scenarios. In particular, the kernel-based similarities are iteratively learned on the deep network features, and the algorithm can be implemented in an end-to-end manner. Extensive experiments are conducted on four benchmark datasets, including MNIST, SVHN, CIFAR-10 and CIFAR-100. The results indicate that KLN achieves state-of-the-art classification performance.

Related papers

• CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics [9.637975962527511]
  We propose a novel generative adversarial network (GAN) variant based on an integral probability metrics framework with characteristic kernel (CKIPM) CKGAN mitigates the notorious problem of mode collapse by mapping the generated images back to random noise. The experimental evaluation conducted on a set of synthetic and real image benchmarks (MNIST, CelebA, etc.) demonstrates that CKGAN generally outperforms other MMD-based GANs.
  arXiv  Detail & Related papers  (2025-04-08T11:58:56Z)
• Continual Learning Using a Kernel-Based Method Over Foundation Models [13.315292874389735]
  Class-incremental learning (CIL) learns a sequence of tasks incrementally. CIL has two key challenges: catastrophic forgetting (CF) and inter-task class separation (ICS) This paper proposes Kernel Linear Discriminant Analysis (KLDA) that can effectively avoid CF and ICS problems.
  arXiv  Detail & Related papers  (2024-12-20T05:09:18Z)
• MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence [97.93517982908007]
  In cross-domain few-shot classification, NCC aims to learn representations to construct a metric space where few-shot classification can be performed. In this paper, we find that there exist high similarities between NCC-learned representations of two samples from different classes. We propose a bi-level optimization framework, emphmaximizing optimized kernel dependence (MOKD) to learn a set of class-specific representations that match the cluster structures indicated by labeled data.
  arXiv  Detail & Related papers  (2024-05-29T05:59:52Z)
• Joint Embedding Self-Supervised Learning in the Kernel Regime [21.80241600638596]
  Self-supervised learning (SSL) produces useful representations of data without access to any labels for classifying the data. We extend this framework to incorporate algorithms based on kernel methods where embeddings are constructed by linear maps acting on the feature space of a kernel. We analyze our kernel model on small datasets to identify common features of self-supervised learning algorithms and gain theoretical insights into their performance on downstream tasks.
  arXiv  Detail & Related papers  (2022-09-29T15:53:19Z)
• Local Sample-weighted Multiple Kernel Clustering with Consensus Discriminative Graph [73.68184322526338]
  Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. This paper proposes a novel local sample-weighted multiple kernel clustering model. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algo-rithms.
  arXiv  Detail & Related papers  (2022-07-05T05:00:38Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Learning with convolution and pooling operations in kernel methods [8.528384027684192]
  Recent empirical work has shown that hierarchical convolutional kernels improve the performance of kernel methods in image classification tasks. We study the precise interplay between approximation and generalization in convolutional architectures. Our results quantify how choosing an architecture adapted to the target function leads to a large improvement in the sample complexity.
  arXiv  Detail & Related papers  (2021-11-16T09:00:44Z)
• Random Features for the Neural Tangent Kernel [57.132634274795066]
  We propose an efficient feature map construction of the Neural Tangent Kernel (NTK) of fully-connected ReLU network. We show that dimension of the resulting features is much smaller than other baseline feature map constructions to achieve comparable error bounds both in theory and practice.
  arXiv  Detail & Related papers  (2021-04-03T09:08:12Z)
• Domain Adaptive Learning Based on Sample-Dependent and Learnable Kernels [2.1485350418225244]
  This paper proposes a Domain Adaptive Learning method based on Sample-Dependent and Learnable Kernels (SDLK-DAL) The first contribution of our work is to propose a sample-dependent and learnable Positive Definite Quadratic Kernel function (PDQK) framework. We conduct a series of experiments that the RKHS determined by PDQK replaces those in several state-of-the-art DAL algorithms, and our approach achieves better performance.
  arXiv  Detail & Related papers  (2021-02-18T13:55:06Z)
• Graph-Aided Online Multi-Kernel Learning [12.805267089186533]
  This paper studies data-driven selection of kernels from the dictionary that provide satisfactory function approximations. Based on the similarities among kernels, the novel framework constructs and refines a graph to assist choosing a subset of kernels. Our proposed algorithms enjoy tighter sub-linear regret bound compared with state-of-art graph-based online MKL alternatives.
  arXiv  Detail & Related papers  (2021-02-09T07:43:29Z)
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data [93.76907759950608]
  We propose a doubly kernel learning algorithm for vertically partitioned data. We show that FDSKL is significantly faster than state-of-the-art federated learning methods when dealing with kernels.
  arXiv  Detail & Related papers  (2020-08-14T05:46:56Z)
• Learning Class Regularized Features for Action Recognition [68.90994813947405]
  We introduce a novel method named Class Regularization that performs class-based regularization of layer activations. We show that using Class Regularization blocks in state-of-the-art CNN architectures for action recognition leads to systematic improvement gains of 1.8%, 1.2% and 1.4% on the Kinetics, UCF-101 and HMDB-51 datasets, respectively.
  arXiv  Detail & Related papers  (2020-02-07T07:27:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.