Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks

• URL: http://arxiv.org/abs/2601.13776v1
• Date: Tue, 20 Jan 2026 09:33:14 GMT
• Title: Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks
• Abstract summary: We introduce Orthogonium, a unified, efficient, and comprehensive PyTorch library providing Orthogonal and 1-Lipschitz layers.<n> Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees.
• Score: 7.578052704279305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Orthogonal and 1-Lipschitz neural network layers are essential building blocks in robust deep learning architectures, crucial for certified adversarial robustness, stable generative models, and reliable recurrent networks. Despite significant advancements, existing implementations remain fragmented, limited, and computationally demanding. To address these issues, we introduce Orthogonium , a unified, efficient, and comprehensive PyTorch library providing orthogonal and 1-Lipschitz layers. Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees. Its optimized implementations reduce overhead on large scale benchmarks such as ImageNet. Moreover, rigorous testing within the library has uncovered critical errors in existing implementations, emphasizing the importance of standardized and reliable tools. Orthogonium thus significantly lowers adoption barriers, enabling scalable experimentation and integration across diverse applications requiring orthogonality and robust Lipschitz constraints. Orthogonium is available at https://github.com/deel-ai/orthogonium.

Related papers

• Möbius-Guided Diagonal-Gate Compilation with Native Multiqubit Controlled-Phase Gates on Neutral-Atom Processors [9.204529285791223]
  We propose a Mbius-guided compiler that maps a diagonal phase function to a phase hypergraph via subset-lattice Mbius inversion.<n>The hypergraph retains the support and angle of each many-body phase term, allowing sparse or local high-order structure to be routed as native multiqubit controlled-phase candidates.
  arXiv  Detail & Related papers  (2026-07-09T08:09:28Z)
• Routing Techniques for Error Corrected Silicon Spin Qubit Quantum Architectures [5.037910041860836]
  Silicon spin qubits have emerged as a promising qubit technology due to their favorable scaling and fabrication properties.<n> efficiently compiling quantum circuits onto spin qubit platforms remains challenging.<n>We introduce a compilation framework for spin qubits based on the recent snakes on a plane model, which utilizes a 2D surface code and qubit teleportation to mitigate errors.<n>We evaluate both algorithms across diverse architectural settings and problem sizes, demonstrating that shortest-path methods excel in sparse, low-defect scenarios, while rotation-based approaches perform better in high-density environments.
  arXiv  Detail & Related papers  (2026-07-08T18:05:06Z)
• Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination [70.27578249309103]
  Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as the cornerstone for shaping the remarkable coding abilities of Large Language Models (LLMs)<n>We propose Atomic Decomposition and Recombination (ADR), a novel framework that generates verifiable code tasks via decomposition into atomic elements and controlled recombination.<n>Our work sheds light on a new paradigm for novel code task synthesis and scalable RLVR training.
  arXiv  Detail & Related papers  (2026-05-29T09:29:32Z)
• SpineContextResUNet: A Computationally Efficient Residual UNet for Spine CT Segmentation [0.21485350418225244]
  We introduce SpineContextResUNet, a computationally efficient 3D Residual U-Net for rapid spinal localization.<n>Our architecture integrates a lightweight Context Block that employs parallel multi-dilated convolutions to capture long-range anatomical dependencies.<n>Our model performs robust inference, making it a viable solution for point-of-care diagnostics and deployment on edge platforms.
  arXiv  Detail & Related papers  (2026-05-20T06:00:23Z)
• Doubly Robust Proxy Causal Learning with Neural Mean Embeddings [58.8808794752741]
  We develop a neural doubly robust framework for proxy causal learning with continuous and structured treatments.<n>The framework covers population, heterogeneous, and conditional dose-response functions, yielding full response-curve estimators.<n>Across synthetic and image-valued benchmarks, the proposed estimators outperform existing baselines and single-bridge neural estimators.
  arXiv  Detail & Related papers  (2026-05-10T12:50:25Z)
• Efficient mapping of multi-constraint satisfaction problems to Rydberg platforms [0.040150524643488644]
  We present a hardware-native gadget framework for solving constraint satisfaction problems on Rydberg quantum computing architectures.<n>Our approach introduces a compact $xor_1$ gadget that enforces exactly-one constraints, ubiquitous in optimization.<n>We observe reductions in detuning range of up to $99%$ and savings in atom count and connectivity overhead.
  arXiv  Detail & Related papers  (2026-04-29T15:01:02Z)
• RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs [5.782015253162346]
  Residual binarization enables matmul-free inference by stacking binary layers.<n>We propose RaBiT, a novel quantization framework that resolves coadaptation by algorithmically enforcing a residual hierarchy.<n>RaBiT achieves state-of-the-art performance, rivals even hardware-intensive Vector Quantization (VQ) methods, and delivers a $4.49times$ inference speed-up over full-precision models.
  arXiv  Detail & Related papers  (2026-02-05T06:41:11Z)
• Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration [52.82397287366076]
  All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework.<n>In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information.<n>Our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections.
  arXiv  Detail & Related papers  (2025-12-11T12:20:31Z)
• Scale-Insensitive Neural Network Significance Tests [0.0]
  This paper develops a scale-insensitive framework for neural network significance testing.<n>We replace metric entropy calculations with Rademacher complexity bounds.<n>We weaken the regularity conditions on the target function to require only Sobolev space membership.
  arXiv  Detail & Related papers  (2025-01-27T03:45:26Z)
• An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures [8.136541584281987]
  We introduce AOC (Adaptative Orthogonal Convolution), a scalable method that extends a previous method (BCOP)<n>We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale.<n>To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium.
  arXiv  Detail & Related papers  (2025-01-14T08:32:12Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Optimal Transport-inspired Deep Learning Framework for Slow-Decaying Kolmogorov n-width Problems: Exploiting Sinkhorn Loss and Wasserstein Kernel [0.0]
  Reduced order models (ROMs) are widely used in scientific computing to tackle high-dimensional systems.<n>We propose a novel ROM framework that integrates optimal transport theory and neural network-based methods.<n>Our framework outperforms traditional ROM methods in terms of accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2023-08-26T10:24:43Z)
• Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration [122.51142131506639]
  We introduce a precise, fast, and differentiable upper bound for the spectral norm of convolutional layers using circulant matrix theory. We show through a comprehensive set of experiments that our approach outperforms other state-of-the-art methods in terms of precision, computational cost, and scalability. It proves highly effective for the Lipschitz regularization of convolutional neural networks, with competitive results against concurrent approaches.
  arXiv  Detail & Related papers  (2023-05-25T15:32:21Z)
• Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach [64.23331120621118]
  This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks. We introduce a related embedded network and show that the embedded network can be used to provide an $ell_infty$-norm box over-approximation of the reachable sets of the original network. We apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.
  arXiv  Detail & Related papers  (2022-08-08T03:13:24Z)
• Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization [44.54772242784423]
  We develop an efficient non regularization algorithm for low-rank learning matrix. The proposed algorithm can avoid expensive folding/unfolding problems. Experiments show that the proposed algorithm is efficient and more space than the existing state-of-the-world.
  arXiv  Detail & Related papers  (2022-05-06T07:47:10Z)
• Orthogonalizing Convolutional Layers with the Cayley Transform [83.73855414030646]
  We propose and evaluate an alternative approach to parameterize convolutional layers that are constrained to be orthogonal. We show that our method indeed preserves orthogonality to a high degree even for large convolutions.
  arXiv  Detail & Related papers  (2021-04-14T23:54:55Z)
• Adaptive Linear Span Network for Object Skeleton Detection [56.78705071830965]
  We propose adaptive linear span network (AdaLSN) to automatically configure and integrate scale-aware features for object skeleton detection. AdaLSN substantiates its versatility by achieving significantly higher accuracy and latency trade-off. It also demonstrates general applicability to image-to-mask tasks such as edge detection and road extraction.
  arXiv  Detail & Related papers  (2020-11-08T12:51:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.