Expressivity of congruence-based architectures for DNNs on positive-definite matrices

• URL: http://arxiv.org/abs/2606.02490v1
• Date: Mon, 01 Jun 2026 17:01:08 GMT
• Title: Expressivity of congruence-based architectures for DNNs on positive-definite matrices
• Abstract summary: We focus on congruence-like layers, in which the input matrix is multiplied on the left and right by a (possibly rectangular) weight matrix $W$ and its transpose.<n>Such layers lie at the core of the celebrated SPDNet and have also been employed independently for dimensionality reduction on positive-definite data.<n>We show that the (semi)-orthogonality constraint commonly imposed on $W$ limits the expressivity of these layers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work studies neural architectures for classifying symmetric positive-definite matrices, focusing on congruence-like layers, in which the input matrix is multiplied on the left and right by a (possibly rectangular) weight matrix $W$ and its transpose. Such layers lie at the core of the celebrated SPDNet and have also been employed independently for dimensionality reduction on positive-definite data. We show that the (semi)-orthogonality constraint commonly imposed on $W$ limits the expressivity of these layers: for certain activation functions, the resulting architecture collapses to a one-hidden-layer equivalent. This lack of expressivity follows from a loss of spectral diversity in congruence-like layers for semi-orthogonal $W$ and is a direct consequence of Poincaré's separation theorem. We then examine the choice of the final classifier, comparing several Riemannian classifiers and discussing their compatibility with the feature maps produced by congruence-like layers.

Related papers

• Riemannian Networks over Full-Rank Correlation Matrices [102.84784139051526]
  Full-rank correlation matrices are a normalized alternative to the Symmetric Positive Definite (SPD) manifold.<n>We extend basic layers, including Multinomial Logistic Regression (MLR), Fully Connected (FC) and convolutional layers, to these geometries.<n>We also present methods for accurate backpropagation for two correlation geometries.
  arXiv  Detail & Related papers  (2026-05-18T19:54:55Z)
• Tree-Preconditioned Differentiable Optimization and Axioms as Layers [0.0]
  "Axioms-as-Layers" paradigm embeds axiomatic structure of Random Utility Models directly into deep neural networks.<n>"Axioms-as-Layers" paradigm eliminates the structural overfitting inherent in penalty-based methods.
  arXiv  Detail & Related papers  (2025-12-03T04:47:37Z)
• Provable Non-Convex Euclidean Distance Matrix Completion: Geometry, Reconstruction, and Robustness [8.113729514518495]
  The Euclidean Distance Matrix Completion problem arises in a broad range of applications, including sensor network localization, molecular robustness, and manifold learning.<n>In this paper, we propose a low-rank matrix completion task over the space of positive semi-definite Gram matrices.<n>The available distance measurements are encoded as expansion coefficients in a non-orthogonal basis, and optimization over the Gram matrix implicitly enforces geometric consistency through nonnegativity and the triangle inequality.
  arXiv  Detail & Related papers  (2025-07-31T18:40:42Z)
• Higher-Order Singular-Value Derivatives of Rectangular Real Matrices [1.5484595752241122]
  We present a theoretical framework for deriving the general $n$-th order Fr'echet derivatives of singular values in real rectangular matrices.<n>We leverage reduced resolvent operators from Kato's perturbation analytic theory for self-adjoint operators.<n>Our framework equips researchers with a practical toolkit for higher-order spectral sensitivity studies in random matrix applications.
  arXiv  Detail & Related papers  (2025-06-04T09:28:35Z)
• Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models [15.650005330621148]
  This work introduces Structured Linear Controlled Differential Equations (SLiCEs)<n>It is a unifying framework for sequence models with structured, input-dependent state-transition matrices.<n>We prove that SLiCEs employ block-diagonal, sparse, or Walsh-Hadamard matrices.
  arXiv  Detail & Related papers  (2025-05-23T11:34:21Z)
• Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
  Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.<n>This paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective.
  arXiv  Detail & Related papers  (2024-07-15T07:11:44Z)
• Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification [72.77513633290056]
  We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model. Our method captures intricate patterns and relationships, enhancing classification performance.
  arXiv  Detail & Related papers  (2024-02-14T16:10:42Z)
• Semi-Supervised Subspace Clustering via Tensor Low-Rank Representation [64.49871502193477]
  We propose a novel semi-supervised subspace clustering method, which is able to simultaneously augment the initial supervisory information and construct a discriminative affinity matrix. Comprehensive experimental results on six commonly-used benchmark datasets demonstrate the superiority of our method over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-21T01:47:17Z)
• Unfolding Projection-free SDP Relaxation of Binary Graph Classifier via GDPA Linearization [59.87663954467815]
  Algorithm unfolding creates an interpretable and parsimonious neural network architecture by implementing each iteration of a model-based algorithm as a neural layer. In this paper, leveraging a recent linear algebraic theorem called Gershgorin disc perfect alignment (GDPA), we unroll a projection-free algorithm for semi-definite programming relaxation (SDR) of a binary graph. Experimental results show that our unrolled network outperformed pure model-based graph classifiers, and achieved comparable performance to pure data-driven networks but using far fewer parameters.
  arXiv  Detail & Related papers  (2021-09-10T07:01:15Z)
• Orthogonalizing Convolutional Layers with the Cayley Transform [83.73855414030646]
  We propose and evaluate an alternative approach to parameterize convolutional layers that are constrained to be orthogonal. We show that our method indeed preserves orthogonality to a high degree even for large convolutions.
  arXiv  Detail & Related papers  (2021-04-14T23:54:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.