VINA: Variational Invertible Neural Architectures

• URL: http://arxiv.org/abs/2602.20480v1
• Date: Tue, 24 Feb 2026 02:16:44 GMT
• Title: VINA: Variational Invertible Neural Architectures
• Abstract summary: We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions.<n>We derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs.<n>We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.
• Score: 11.068746821872756
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural networks (INNs) build on these principles to address supervised inverse problems, enabling direct modeling of both forward and inverse mappings. In this paper, we revisit these architectures from both theoretical and practical perspectives and address a key gap in the literature: the lack of theoretical guarantees on approximation quality under realistic assumptions, whether for posterior inference in INNs or for generative modeling with NFs. We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions, inspired by analogous formulations in related areas such as generative adversarial networks (GANs) and the Precision-Recall divergence for training normalizing flows. Within this framework, we derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs, under assumptions that are weaker and more practically realistic than those used in prior work. Building on these theoretical results, we conduct extensive case studies to distill general design principles and practical guidelines. We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.

Related papers

• Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks [0.0]
  Deep neural network (DNN) training with gradient descent (SGD) and its variants achieves strong empirical performance.<n>Traditional analyses rely on assumptions such as differentiability, convexity, or smoothness.<n>We introduce $mathcalH()$ smoothness and $mathcalH()$-smoothness, which unify convex non-smooth as well as smooth non-smooth objectives.
  arXiv  Detail & Related papers  (2026-08-10T12:24:29Z)
• A Theory on Flow Matching with Neural Networks [18.21009159346104]
  We develop theoretical foundation for flow matching with conditional velocity fields.<n>We derive generalization bounds for the conditional velocity-field matching objective.<n>Our analysis is based on generalization bound for multi-task representation learning with unbounded losses.
  arXiv  Detail & Related papers  (2026-06-08T19:16:32Z)
• From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips [0.6868507883617846]
  In this work, we study resilience as a structural property of neural architectures.<n>We show that lower precision, higher sparsity, bounded activations, and shallow depth are consistently favored.<n>We identify a novel even-layer recovery effect unique to logic-based architectures.
  arXiv  Detail & Related papers  (2026-03-24T03:53:04Z)
• Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks [0.0]
  We develop a conjugate learning theoretical framework based on convex conjugate duality to characterize this learnability property.<n>We demonstrate that training deep neural networks (DNNs) with mini-batch descent (SGD) achieves global optima of empirical risk.<n>We derive deterministic and probabilistic bounds on generalization error based on conditional generalized entropy measures.
  arXiv  Detail & Related papers  (2026-02-18T04:26:55Z)
• Deep Neural Networks as Iterated Function Systems and a Generalization Bound [2.7920304852537536]
  We show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS.<n>We derive a Wasserstein bound for generative modeling that controls the collage-type approximation error between the data distribution and its image.
  arXiv  Detail & Related papers  (2026-01-27T07:32:49Z)
• Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds [63.47271262149291]
  We propose a unified framework for PAC-Bayesian norm-based generalization.<n>The key to our approach is a sensitivity matrix that quantifies the network outputs with respect to structured weight perturbations.<n>We derive a family of generalization bounds that recover several existing PAC-Bayesian results as special cases.
  arXiv  Detail & Related papers  (2026-01-13T00:42:22Z)
• From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing [0.0]
  The study covers foundational models such as Deep Operator Networks (DeepONet) and Principal Component Analysis-based Neural Networks (PCANet)<n>The review delves into applying neural operators as surrogates in Bayesian inference problems, showcasing their effectiveness in accelerating posterior inference while maintaining accuracy.<n>It outlines emerging strategies to address these issues, such as residual-based error correction and multi-level training.
  arXiv  Detail & Related papers  (2025-03-07T17:25:25Z)
• ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks [71.02216400133858]
  Physics-informed neural networks (PINNs) have earned high expectations in solving partial differential equations (PDEs)<n>Previous research observed the propagation failure phenomenon of PINNs.<n>This paper provides a formal and in-depth study of propagation failure and its root cause.
  arXiv  Detail & Related papers  (2025-02-02T13:56:38Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective [9.753461673117362]
  The neural tangent kernel (NTK) has garnered significant attention as a theoretical framework for describing the behavior of large-scale neural networks. Current results quantifying the rate of convergence to the kernel regime suggest that exploiting these benefits requires architectures that are orders of magnitude wider than they are deep. This paper investigates whether the limiting regime predicts practically relevant behavior of large-width architectures.
  arXiv  Detail & Related papers  (2023-09-29T20:51:24Z)
• Generalization and Estimation Error Bounds for Model-based Neural Networks [78.88759757988761]
  We show that the generalization abilities of model-based networks for sparse recovery outperform those of regular ReLU networks. We derive practical design rules that allow to construct model-based networks with guaranteed high generalization.
  arXiv  Detail & Related papers  (2023-04-19T16:39:44Z)
• Extended critical regimes of deep neural networks [0.0]
  We show that heavy-tailed weights enable the emergence of an extended critical regime without fine-tuning parameters. In this extended critical regime, DNNs exhibit rich and complex propagation dynamics across layers. We provide a theoretical guide for the design of efficient neural architectures.
  arXiv  Detail & Related papers  (2022-03-24T10:15:50Z)
• A Theoretical Framework for Target Propagation [75.52598682467817]
  We analyze target propagation (TP), a popular but not yet fully understood alternative to backpropagation (BP) Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training.
  arXiv  Detail & Related papers  (2020-06-25T12:07:06Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.