Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks

• URL: http://arxiv.org/abs/2601.13776v1
• Date: Tue, 20 Jan 2026 09:33:14 GMT
• Title: Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks
• Authors: Thibaut Boissin, Franck Mamalet, Valentin Lafargue, Mathieu Serrurier,
• Abstract summary: We introduce Orthogonium, a unified, efficient, and comprehensive PyTorch library providing Orthogonal and 1-Lipschitz layers.<n> Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees.
• Score: 7.578052704279305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Orthogonal and 1-Lipschitz neural network layers are essential building blocks in robust deep learning architectures, crucial for certified adversarial robustness, stable generative models, and reliable recurrent networks. Despite significant advancements, existing implementations remain fragmented, limited, and computationally demanding. To address these issues, we introduce Orthogonium , a unified, efficient, and comprehensive PyTorch library providing orthogonal and 1-Lipschitz layers. Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees. Its optimized implementations reduce overhead on large scale benchmarks such as ImageNet. Moreover, rigorous testing within the library has uncovered critical errors in existing implementations, emphasizing the importance of standardized and reliable tools. Orthogonium thus significantly lowers adoption barriers, enabling scalable experimentation and integration across diverse applications requiring orthogonality and robust Lipschitz constraints. Orthogonium is available at https://github.com/deel-ai/orthogonium.

Related papers

• RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs [5.782015253162346]
  Residual binarization enables matmul-free inference by stacking binary layers.<n>We propose RaBiT, a novel quantization framework that resolves coadaptation by algorithmically enforcing a residual hierarchy.<n>RaBiT achieves state-of-the-art performance, rivals even hardware-intensive Vector Quantization (VQ) methods, and delivers a $4.49times$ inference speed-up over full-precision models.
  arXiv  Detail & Related papers  (2026-02-05T06:41:11Z)
• Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration [52.82397287366076]
  All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework.<n>In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information.<n>Our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections.
  arXiv  Detail & Related papers  (2025-12-11T12:20:31Z)
• Scale-Insensitive Neural Network Significance Tests [0.0]
  This paper develops a scale-insensitive framework for neural network significance testing.<n>We replace metric entropy calculations with Rademacher complexity bounds.<n>We weaken the regularity conditions on the target function to require only Sobolev space membership.
  arXiv  Detail & Related papers  (2025-01-27T03:45:26Z)
• An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures [8.136541584281987]
  We introduce AOC (Adaptative Orthogonal Convolution), a scalable method that extends a previous method (BCOP)<n>We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale.<n>To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium.
  arXiv  Detail & Related papers  (2025-01-14T08:32:12Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Optimal Transport-inspired Deep Learning Framework for Slow-Decaying Kolmogorov n-width Problems: Exploiting Sinkhorn Loss and Wasserstein Kernel [0.0]
  Reduced order models (ROMs) are widely used in scientific computing to tackle high-dimensional systems.<n>We propose a novel ROM framework that integrates optimal transport theory and neural network-based methods.<n>Our framework outperforms traditional ROM methods in terms of accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2023-08-26T10:24:43Z)
• Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration [122.51142131506639]
  We introduce a precise, fast, and differentiable upper bound for the spectral norm of convolutional layers using circulant matrix theory. We show through a comprehensive set of experiments that our approach outperforms other state-of-the-art methods in terms of precision, computational cost, and scalability. It proves highly effective for the Lipschitz regularization of convolutional neural networks, with competitive results against concurrent approaches.
  arXiv  Detail & Related papers  (2023-05-25T15:32:21Z)
• Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach [64.23331120621118]
  This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks. We introduce a related embedded network and show that the embedded network can be used to provide an $ell_infty$-norm box over-approximation of the reachable sets of the original network. We apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.
  arXiv  Detail & Related papers  (2022-08-08T03:13:24Z)
• Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization [44.54772242784423]
  We develop an efficient non regularization algorithm for low-rank learning matrix. The proposed algorithm can avoid expensive folding/unfolding problems. Experiments show that the proposed algorithm is efficient and more space than the existing state-of-the-world.
  arXiv  Detail & Related papers  (2022-05-06T07:47:10Z)
• Orthogonalizing Convolutional Layers with the Cayley Transform [83.73855414030646]
  We propose and evaluate an alternative approach to parameterize convolutional layers that are constrained to be orthogonal. We show that our method indeed preserves orthogonality to a high degree even for large convolutions.
  arXiv  Detail & Related papers  (2021-04-14T23:54:55Z)
• Adaptive Linear Span Network for Object Skeleton Detection [56.78705071830965]
  We propose adaptive linear span network (AdaLSN) to automatically configure and integrate scale-aware features for object skeleton detection. AdaLSN substantiates its versatility by achieving significantly higher accuracy and latency trade-off. It also demonstrates general applicability to image-to-mask tasks such as edge detection and road extraction.
  arXiv  Detail & Related papers  (2020-11-08T12:51:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.