Divergence Regulated Encoder Network for Joint Dimensionality Reduction and Classification

• URL: http://arxiv.org/abs/2012.15764v3
• Date: Tue, 16 Mar 2021 20:03:12 GMT
• Title: Divergence Regulated Encoder Network for Joint Dimensionality Reduction and Classification
• Authors: Joshua Peeples, Sarah Walker, Connor McCurley, Alina Zare, James Keller
• Abstract summary: We investigate performing joint dimensionality reduction and classification using a novel histogram neural network. Motivated by a popular dimensionality reduction approach, t-Distributed Neighbor Embedding (t-SNE), our proposed method incorporates a classification loss computed on samples in a low-dimensional embedding space. Our results show that the proposed approach maintains and/or improves classification performance and reveals characteristics of features produced by neural networks that may be helpful for other applications.
• Score: 2.989889278970106
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we investigate performing joint dimensionality reduction and classification using a novel histogram neural network. Motivated by a popular dimensionality reduction approach, t-Distributed Stochastic Neighbor Embedding (t-SNE), our proposed method incorporates a classification loss computed on samples in a low-dimensional embedding space. We compare the learned sample embeddings against coordinates found by t-SNE in terms of classification accuracy and qualitative assessment. We also explore use of various divergence measures in the t-SNE objective. The proposed method has several advantages such as readily embedding out-of-sample points and reducing feature dimensionality while retaining class discriminability. Our results show that the proposed approach maintains and/or improves classification performance and reveals characteristics of features produced by neural networks that may be helpful for other applications.

Related papers

• Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation [57.10353686244835]
  We address the problem of verifying neural networks against geometric transformations of the input image, including rotation, scaling, shearing, and translation. The proposed method computes provably sound piecewise linear constraints for the pixel values by using sampling and linear approximations in combination with branch-and-bound Lipschitz. We show that our proposed implementation resolves up to 32% more verification cases than present approaches.
  arXiv  Detail & Related papers  (2024-08-23T15:02:09Z)
• Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric [99.19559537966538]
  DML aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-03T13:44:20Z)
• A Mallows-like Criterion for Anomaly Detection with Random Forest Implementation [7.569443648362081]
  This paper proposes a novel criterion to select the weights on aggregation of multiple models, wherein the focal loss function accounts for the classification of extremely imbalanced data. We have evaluated the proposed method on benchmark datasets across various domains, including network intrusion.
  arXiv  Detail & Related papers  (2024-05-29T09:36:57Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• A Systematic Evaluation of Node Embedding Robustness [77.29026280120277]
  We assess the empirical robustness of node embedding models to random and adversarial poisoning attacks. We compare edge addition, deletion and rewiring strategies computed using network properties as well as node labels. We found that node classification suffers from higher performance degradation as opposed to network reconstruction.
  arXiv  Detail & Related papers  (2022-09-16T17:20:23Z)
• Supervised Dimensionality Reduction and Classification with Convolutional Autoencoders [1.1164202369517053]
  A Convolutional Autoencoder is combined to simultaneously produce supervised dimensionality reduction and predictions. The resulting Latent Space can be utilized to improve traditional, interpretable classification algorithms. The proposed methodology introduces advanced explainability regarding, not only the data structure through the produced latent space, but also about the classification behaviour.
  arXiv  Detail & Related papers  (2022-08-25T15:18:33Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks [11.634729459989996]
  We introduce emerging information theoretic feature transformation protocols as an end-to-end neural network training approach. We present a dimensionality reduction network (MMINet) training procedure based on the estimate of the mutual information gradient. We experimentally evaluate our method with applications to high-dimensional biological data sets, and relate it to conventional feature selection algorithms.
  arXiv  Detail & Related papers  (2021-05-01T08:20:04Z)
• Joint Dimensionality Reduction for Separable Embedding Estimation [43.22422640265388]
  Low-dimensional embeddings for data from disparate sources play critical roles in machine learning, multimedia information retrieval, and bioinformatics. We propose a supervised dimensionality reduction method that learns linear embeddings jointly for two feature vectors representing data of different modalities or data from distinct types of entities. Our approach compares favorably against other dimensionality reduction methods, and against a state-of-the-art method of bilinear regression for predicting gene-disease associations.
  arXiv  Detail & Related papers  (2021-01-14T08:48:37Z)
• Dimensionality Reduction via Diffusion Map Improved with Supervised Linear Projection [1.7513645771137178]
  In this paper, we assume the data samples lie on a single underlying smooth manifold. We define intra-class and inter-class similarities using pairwise local kernel distances. We aim to find a linear projection to maximize the intra-class similarities and minimize the inter-class similarities simultaneously.
  arXiv  Detail & Related papers  (2020-08-08T04:26:07Z)
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification [49.87503122462432]
  We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet) Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms. Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
  arXiv  Detail & Related papers  (2020-06-27T13:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.