CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks

• URL: http://arxiv.org/abs/2605.12580v1
• Date: Tue, 12 May 2026 15:46:23 GMT
• Title: CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks
• Abstract summary: We propose CAWI (Copula-Aligned Weight Initialization), a framework that draws input-to-hidden weights from a data-fitted copula.<n>We evaluate CAWI across 83 diverse classification benchmarks and two biomedical datasets.
• Score: 7.698602367824994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Randomized neural networks (RdNNs) enable efficient, backpropagation-free training by freezing randomly initialized input-to-hidden weights, which permits a closed-form solution for the output layer. However, conventional random initialization is blind to inter-feature dependence, ignoring correlations, asymmetries, and tail dependence in the data, which degrades conditioning and predictive performance. To the best of our knowledge, this limitation remains unaddressed in the RdNN literature. To close this gap, we propose CAWI (Copula-Aligned Weight Initialization), a framework that draws input-to-hidden weights from a data-fitted copula that matches empirical dependence, ensuring the frozen projections respect inter-feature dependence without sacrificing the closed-form solution. CAWI (i) maps each feature to the unit interval using empirical CDFs, (ii) fits a multivariate copula that captures rank-based dependence among features, and (iii) samples each weight column w_j from the fitted copula and applies a fixed inverse marginal transform to set scale. The objective, solver, and "freeze-once" paradigm remain unchanged; only the sampling law for W becomes dependence-aware. For dependence modeling, we consider two copula families: elliptical (Gaussian, t) and Archimedean (Clayton, Frank, Gumbel). This enables CAWI to handle diverse dependence, including tail dependence. We evaluate CAWI across 83 diverse classification benchmarks (binary and multiclass) and two biomedical datasets, BreaKHis and the Schizophrenia dataset, using standard shallow and deep RdNN architectures. CAWI consistently delivers significant improvements in predictive performance over conventional random initialization. Code is available at: https://github.com/mtanveer1/CAWI

Related papers

• Adaptive deep nonparametric regression from dependent data under covariate shift [0.0]
  We deal with a generalized Bernstein-type inequality that is satisfied by many classical models.<n>We propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data.<n>For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established.
  arXiv  Detail & Related papers  (2026-07-22T15:55:46Z)
• A Mutual Information Lower Bound for Multimodal Regression Active Learning [9.04041860173466]
  Mutual Information Lower Bound (MI-LB) is a closed-form approximation for Mixture Network ensembles.<n>We show that MI-LB matches or beats every baseline evaluated on benchmarks featuring multimodal systems.
  arXiv  Detail & Related papers  (2026-05-14T14:50:47Z)
• Factual recall in linear associative memories: sharp asymptotics and mechanistic insights [11.11958822145404]
  We study the limits of storing and retrieving input--output associations with neural networks.<n>A linear associative memory maps $p$ input embeddings in $mathbbRd$ to their corresponding$d$-dimensional targets.<n>We show that a decoupled model can store up to $p_c log p_c / d2 = 1 / 2$ associations.
  arXiv  Detail & Related papers  (2026-05-11T16:24:02Z)
• Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction [0.26856688022781555]
  We introduce Dual-r, which generalizes multistep samplers through learnable parameters.<n>It retains the standard predictor-corrector structure while preserving second-order local accuracy.<n>It improves FID and CLIP scores in the low-NFE regime across backbones.
  arXiv  Detail & Related papers  (2026-03-04T12:14:52Z)
• Self-supervised Latent Space Optimization with Nebula Variational Coding [87.20343320266215]
  This paper proposes a variational inference model which leads to a clustered embedding.<n>We introduce additional variables in the latent space, called textbfnebula anchors, that guide the latent variables to form clusters during training.<n>Since each latent feature can be labeled with the closest anchor, we also propose to apply metric learning in a self-supervised way to make the separation between clusters more explicit.
  arXiv  Detail & Related papers  (2025-06-02T08:13:32Z)
• Neural variational Data Assimilation with Uncertainty Quantification using SPDE priors [28.804041716140194]
  Recent advances in the deep learning community enables to address the problem through a neural architecture a variational data assimilation framework.<n>In this work we use the theory of Partial Differential Equations (SPDE) and Gaussian Processes (GP) to estimate both space-and time covariance of the state.
  arXiv  Detail & Related papers  (2024-02-02T19:18:12Z)
• CeCNN: Copula-enhanced convolutional neural networks in joint prediction of refraction error and axial length based on ultra-widefield fundus images [6.787893694522311]
  We propose the Copula-enhanced Convolutional Neural Network (CeCNN) to jointly predict Spherical Equivalence (SE) measurement and high myopia diagnosis.
  arXiv  Detail & Related papers  (2023-11-07T13:06:50Z)
• FedNAR: Federated Optimization with Normalized Annealing Regularization [54.42032094044368]
  We explore the choices of weight decay and identify that weight decay value appreciably influences the convergence of existing FL algorithms. We develop Federated optimization with Normalized Annealing Regularization (FedNAR), a plug-in that can be seamlessly integrated into any existing FL algorithms.
  arXiv  Detail & Related papers  (2023-10-04T21:11:40Z)
• Sparse-Input Neural Network using Group Concave Regularization [10.103025766129006]
  Simultaneous feature selection and non-linear function estimation are challenging in neural networks. We propose a framework of sparse-input neural networks using group concave regularization for feature selection in both low-dimensional and high-dimensional settings.
  arXiv  Detail & Related papers  (2023-07-01T13:47:09Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.