Improvising the Learning of Neural Networks on Hyperspherical Manifold

• URL: http://arxiv.org/abs/2109.14746v1
• Date: Wed, 29 Sep 2021 22:39:07 GMT
• Title: Improvising the Learning of Neural Networks on Hyperspherical Manifold
• Authors: Lalith Bharadwaj Baru, Sai Vardhan Kanumolu, Akshay Patel Shilhora
• Abstract summary: The impact of convolution neural networks (CNNs) in the supervised settings provided tremendous increment in performance. The representation learned from CNN's operated on hyperspherical manifold led to insightful outcomes in face recognition. A broad range of activation functions is developed with hypersphere intuition which performs superior to softmax in euclidean space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The impact of convolution neural networks (CNNs) in the supervised settings provided tremendous increment in performance. The representations learned from CNN's operated on hyperspherical manifold led to insightful outcomes in face recognition, face identification and other supervised tasks. A broad range of activation functions is developed with hypersphere intuition which performs superior to softmax in euclidean space. The main motive of this research is to provide insights. First, the stereographic projection is implied to transform data from Euclidean space ($\mathbb{R}^{n}$) to hyperspherical manifold ($\mathbb{S}^{n}$) to analyze the performance of angular margin losses. Secondly, proving both theoretically and practically that decision boundaries constructed on hypersphere using stereographic projection obliges the learning of neural networks. Experiments have proved that applying stereographic projection on existing state-of-the-art angular margin objective functions led to improve performance for standard image classification data sets (CIFAR-10,100). The code is publicly available at: https://github.com/barulalithb/stereo-angular-margin.

Related papers

• On the Universal Statistical Consistency of Expansive Hyperbolic Deep Convolutional Neural Networks [14.904264782690639]
  In this work, we propose Hyperbolic DCNN based on the Poincar'e Disc. We offer extensive theoretical insights pertaining to the universal consistency of the expansive convolution in the hyperbolic space. Results reveal that the hyperbolic convolutional architecture outperforms the Euclidean ones by a commendable margin.
  arXiv  Detail & Related papers  (2024-11-15T12:01:03Z)
• Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
  This paper introduces the SANE approach to weight-space learning. Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
  arXiv  Detail & Related papers  (2024-06-14T13:12:07Z)
• Parameter Convex Neural Networks [13.42851919291587]
  We propose the exponential multilayer neural network (EMLP) which is convex with regard to the parameters of the neural network under some conditions. For late experiments, we use the same architecture to make the exponential graph convolutional network (EGCN) and do the experiment on the graph classificaion dataset.
  arXiv  Detail & Related papers  (2022-06-11T16:44:59Z)
• FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks [77.34726150561087]
  We show how to explore high-dimensional landscape characteristics of neural networks. We generalize observations on small neural networks to more complex systems. An interactive dashboard opens up a number of possible application networks.
  arXiv  Detail & Related papers  (2022-04-09T16:41:53Z)
• Optimization-Based Separations for Neural Networks [57.875347246373956]
  We show that gradient descent can efficiently learn ball indicator functions using a depth 2 neural network with two layers of sigmoidal activations. This is the first optimization-based separation result where the approximation benefits of the stronger architecture provably manifest in practice.
  arXiv  Detail & Related papers  (2021-12-04T18:07:47Z)
• Learning A 3D-CNN and Transformer Prior for Hyperspectral Image Super-Resolution [80.93870349019332]
  We propose a novel HSISR method that uses Transformer instead of CNN to learn the prior of HSIs. Specifically, we first use the gradient algorithm to solve the HSISR model, and then use an unfolding network to simulate the iterative solution processes.
  arXiv  Detail & Related papers  (2021-11-27T15:38:57Z)
• Free Hyperbolic Neural Networks with Limited Radii [32.42488915688723]
  Hyperbolic Neural Networks (HNNs) that operate directly in hyperbolic space have been proposed recently to further exploit the potential of hyperbolic representations. While HNNs have achieved better performance than Euclidean neural networks (ENNs) on datasets with implicit hierarchical structure, they still perform poorly on standard classification benchmarks such as CIFAR and ImageNet. In this paper, we first conduct an empirical study showing that the inferior performance of HNNs on standard recognition datasets can be attributed to the notorious vanishing gradient problem. Our analysis leads to a simple yet effective solution called Feature Clipping, which regularizes the hyperbolic embedding whenever its
  arXiv  Detail & Related papers  (2021-07-23T22:10:16Z)
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression [59.58879331876508]
  We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
  arXiv  Detail & Related papers  (2021-07-19T22:14:30Z)
• Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling [79.15521784128102]
  We introduce a novel neural network for building image generators (decoders) and apply it to variational autoencoders (VAEs) In our spatial dependency networks (SDNs), feature maps at each level of a deep neural net are computed in a spatially coherent way. We show that augmenting the decoder of a hierarchical VAE by spatial dependency layers considerably improves density estimation.
  arXiv  Detail & Related papers  (2021-03-16T07:01:08Z)
• Learning Granularity-Aware Convolutional Neural Network for Fine-Grained Visual Classification [0.0]
  We propose a novel Granularity-Aware Congrainedal Neural Network (GA-CNN) that progressively explores discriminative features. GA-CNN does not need bounding boxes/part annotations and can be trained end-to-end. Our approach achieves state-of-the-art performances on three benchmark datasets.
  arXiv  Detail & Related papers  (2021-03-04T02:18:07Z)
• Hyperbolic Generative Adversarial Network [0.0]
  We propose that it is possible to take advantage of the hierarchical characteristic present in the images by using hyperbolic neural networks in a GAN architecture. In this study, different configurations using fully connected hyperbolic layers in the GAN, CGAN, and WGAN are tested, in what we call the HGAN, HCGAN, and HWGAN, respectively.
  arXiv  Detail & Related papers  (2021-02-10T16:55:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.