Hyperbolic Deep Neural Networks: A Survey
- URL: http://arxiv.org/abs/2101.04562v3
- Date: Wed, 17 Feb 2021 14:59:23 GMT
- Title: Hyperbolic Deep Neural Networks: A Survey
- Authors: Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, Guoying
Zhao
- Abstract summary: We refer to the model as hyperbolic deep neural network in this paper.
To stimulate future research, this paper presents acoherent and comprehensive review of the literature around the neural components in the construction of hyperbolic deep neuralnetworks.
- Score: 31.04110049167551
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Recently, there has been a rising surge of momentum for deep representation
learning in hyperbolic spaces due to theirhigh capacity of modeling data like
knowledge graphs or synonym hierarchies, possessing hierarchical structure. We
refer to the model as hyperbolic deep neural network in this paper. Such a
hyperbolic neural architecture potentially leads to drastically compact model
withmuch more physical interpretability than its counterpart in Euclidean
space. To stimulate future research, this paper presents acoherent and
comprehensive review of the literature around the neural components in the
construction of hyperbolic deep neuralnetworks, as well as the generalization
of the leading deep approaches to the Hyperbolic space. It also presents
current applicationsaround various machine learning tasks on several publicly
available datasets, together with insightful observations and identifying
openquestions and promising future directions.
Related papers
- On the Universal Statistical Consistency of Expansive Hyperbolic Deep Convolutional Neural Networks [14.904264782690639]
In this work, we propose Hyperbolic DCNN based on the Poincar'e Disc.
We offer extensive theoretical insights pertaining to the universal consistency of the expansive convolution in the hyperbolic space.
Results reveal that the hyperbolic convolutional architecture outperforms the Euclidean ones by a commendable margin.
arXiv Detail & Related papers (2024-11-15T12:01:03Z) - Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
This paper introduces the SANE approach to weight-space learning.
Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
arXiv Detail & Related papers (2024-06-14T13:12:07Z) - Nonlinear classification of neural manifolds with contextual information [6.292933471495322]
manifold capacity has emerged as a promising framework linking population geometry to the separability of neural manifold.
We propose a theoretical framework that overcomes this limitation by leveraging contextual input information.
Our framework's increased expressivity captures representation untanglement in deep networks at early stages of the layer hierarchy, previously inaccessible to analysis.
arXiv Detail & Related papers (2024-05-10T23:37:31Z) - Hyperbolic Graph Neural Networks: A Review of Methods and Applications [55.5502008501764]
Graph neural networks generalize conventional neural networks to graph-structured data.
The performance of Euclidean models in graph-related learning is still bounded and limited by the representation ability of Euclidean geometry.
Recently, hyperbolic space has gained increasing popularity in processing graph data with tree-like structure and power-law distribution.
arXiv Detail & Related papers (2022-02-28T15:08:48Z) - Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
Deep learning has led to a rising demand for neural network architecture engineering.
neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually.
NAS has become applicable to a much wider range of problems in computer vision.
arXiv Detail & Related papers (2022-02-15T08:06:50Z) - Creating Powerful and Interpretable Models withRegression Networks [2.2049183478692584]
We propose a novel architecture, Regression Networks, which combines the power of neural networks with the understandability of regression analysis.
We demonstrate that the models exceed the state-of-the-art performance of interpretable models on several benchmark datasets.
arXiv Detail & Related papers (2021-07-30T03:37:00Z) - Fully Hyperbolic Neural Networks [63.22521652077353]
We propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model.
We show that our method has better performance for building both shallow and deep networks.
arXiv Detail & Related papers (2021-05-31T03:36:49Z) - Learning Contact Dynamics using Physically Structured Neural Networks [81.73947303886753]
We use connections between deep neural networks and differential equations to design a family of deep network architectures for representing contact dynamics between objects.
We show that these networks can learn discontinuous contact events in a data-efficient manner from noisy observations.
Our results indicate that an idealised form of touch feedback is a key component of making this learning problem tractable.
arXiv Detail & Related papers (2021-02-22T17:33:51Z) - Hyperbolic Neural Networks++ [66.16106727715061]
We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model.
Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
arXiv Detail & Related papers (2020-06-15T08:23:20Z) - An Overview of Neural Network Compression [2.550900579709111]
In recent years there has been a resurgence in model compression techniques, particularly for deep convolutional neural networks and self-attention based networks such as the Transformer.
This paper provides a timely overview of both old and current compression techniques for deep neural networks, including pruning, quantization, tensor decomposition, knowledge distillation and combinations thereof.
arXiv Detail & Related papers (2020-06-05T20:28:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.