Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning

• URL: http://arxiv.org/abs/2405.07024v1
• Date: Sat, 11 May 2024 14:41:48 GMT
• Title: Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning
• Authors: Danilo Comminiello, Eleonora Grassucci, Danilo P. Mandic, Aurelio Uncini,
• Abstract summary: This paper provides a framework for understanding why hypercomplex deep learning methods are so successful and how their potential can be exploited. We show that it is possible to derive specific inductive biases in the hypercomplex domains. These biases prove effective in managing the distinctive properties of these domains, as well as the complex structures of multidimensional and multimodal signals.
• Score: 23.501824517684465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hypercomplex algebras have recently been gaining prominence in the field of deep learning owing to the advantages of their division algebras over real vector spaces and their superior results when dealing with multidimensional signals in real-world 3D and 4D paradigms. This paper provides a foundational framework that serves as a roadmap for understanding why hypercomplex deep learning methods are so successful and how their potential can be exploited. Such a theoretical framework is described in terms of inductive bias, i.e., a collection of assumptions, properties, and constraints that are built into training algorithms to guide their learning process toward more efficient and accurate solutions. We show that it is possible to derive specific inductive biases in the hypercomplex domains, which extend complex numbers to encompass diverse numbers and data structures. These biases prove effective in managing the distinctive properties of these domains, as well as the complex structures of multidimensional and multimodal signals. This novel perspective for hypercomplex deep learning promises to both demystify this class of methods and clarify their potential, under a unifying framework, and in this way promotes hypercomplex models as viable alternatives to traditional real-valued deep learning for multidimensional signal processing.

Related papers

• Learning Divergence Fields for Shift-Robust Graph Representations [73.11818515795761]
  In this work, we propose a geometric diffusion model with learnable divergence fields for the challenging problem with interdependent data. We derive a new learning objective through causal inference, which can guide the model to learn generalizable patterns of interdependence that are insensitive across domains.
  arXiv  Detail & Related papers  (2024-06-07T14:29:21Z)
• Combinatorial Complexes: Bridging the Gap Between Cell Complexes and Hypergraphs [18.793940779717627]
  We argue that hypergraphs and cell complexes emphasize emphdifferent types of relations, which may have different utility depending on the application context. We discuss the relative advantages of these two choices and elaborate on the previously introduced concept of a complex that enables co-existing set-type and hierarchical relations.
  arXiv  Detail & Related papers  (2023-12-15T03:04:28Z)
• Higher-order topological kernels via quantum computation [68.8204255655161]
  Topological data analysis (TDA) has emerged as a powerful tool for extracting meaningful insights from complex data. We propose a quantum approach to defining Betti kernels, which is based on constructing Betti curves with increasing order.
  arXiv  Detail & Related papers  (2023-07-14T14:48:52Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Bounds on quantum evolution complexity via lattice cryptography [0.0]
  We address the difference between integrable and chaotic motion in quantum theory as manifested by the complexity of the corresponding evolution operators. Complexity is understood here as the shortest geodesic distance between the time-dependent evolution operator and the origin within the group of unitaries.
  arXiv  Detail & Related papers  (2022-02-28T16:20:10Z)
• Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions [66.05472746340142]
  Generative networks (GAN) are among the most successful for learning high-complexity, real-world distributions. In this paper we show how GANs can efficiently learn to the distribution of real-life images.
  arXiv  Detail & Related papers  (2021-06-04T17:33:29Z)
• Quadric hypersurface intersection for manifold learning in feature space [52.83976795260532]
  manifold learning technique suitable for moderately high dimension and large datasets. The technique is learned from the training data in the form of an intersection of quadric hypersurfaces. At test time, this manifold can be used to introduce an outlier score for arbitrary new points.
  arXiv  Detail & Related papers  (2021-02-11T18:52:08Z)
• A General Framework for Hypercomplex-valued Extreme Learning Machines [2.055949720959582]
  This paper aims to establish a framework for extreme learning machines (ELMs) on general hypercomplex algebras. We show a framework to operate in these algebras through real-valued linear algebra operations. Experiments highlight the excellent performance of hypercomplex-valued ELMs to treat high-dimensional data.
  arXiv  Detail & Related papers  (2021-01-15T15:22:05Z)
• Learning Theory for Inferring Interaction Kernels in Second-Order Interacting Agent Systems [17.623937769189364]
  We develop a complete learning theory which establishes strong consistency and optimal nonparametric min-max rates of convergence for the estimators. The numerical algorithm presented to build the estimators is parallelizable, performs well on high-dimensional problems, and is demonstrated on complex dynamical systems.
  arXiv  Detail & Related papers  (2020-10-08T02:07:53Z)
• Unitary Learning for Deep Diffractive Neural Network [0.0]
  We present a unitary learning protocol on deep diffractive neural network. The temporal-space evolution characteristic in unitary learning is formulated and elucidated. As a preliminary application, deep diffractive neural network with unitary learning is tentatively implemented on the 2D classification and verification tasks.
  arXiv  Detail & Related papers  (2020-08-17T07:16:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.