Parallel Network Reconstruction with Multi-directional Regularization

• URL: http://arxiv.org/abs/2411.11464v2
• Date: Mon, 18 Aug 2025 19:11:55 GMT
• Title: Parallel Network Reconstruction with Multi-directional Regularization
• Authors: Zhaoyu Xing, Wei Zhong,
• Abstract summary: We introduce a new distributed computing framework for efficient large-scale network reconstruction with parallel computing, namely PALMS (Parallel Adaptive Lasso with Multi-directional Signals).
• Score: 6.949853145487146
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing large-scale latent networks from observed dynamics is crucial for understanding complex systems. However, the existing methods based on compressive sensing are often rendered infeasible in practice by prohibitive computational and memory costs. To address this challenge, we introduce a new distributed computing framework for efficient large-scale network reconstruction with parallel computing, namely PALMS (Parallel Adaptive Lasso with Multi-directional Signals). The core idea of PALMS is to decompose the complex global problem by partitioning network nodes, enabling the parallel estimation of sub-networks across multiple computing units. This strategy substantially reduces the computational complexity and storage requirements of classic methods. By using the adaptive multi-directional regularization on each computing unit, we also establish the consistency of PALMS estimator theoretically. Extensive simulation studies and empirical analyses on several large-scale real-world networks validate the computational efficiency and robust reconstruction accuracy of our approach.

Related papers

• Unlocking Symbol-Level Precoding Efficiency Through Tensor Equivariant Neural Network [84.22115118596741]
  We propose an end-to-end deep learning (DL) framework with low inference complexity for symbol-level precoding.<n>We show that the proposed framework captures substantial performance gains of optimal SLP, while achieving an approximately 80-times speedup over conventional methods.
  arXiv  Detail & Related papers  (2025-10-02T15:15:50Z)
• (Sometimes) Less is More: Mitigating the Complexity of Rule-based Representation for Interpretable Classification [1.6504157612470989]
  Deep neural networks are widely used in practical applications of AI, but their inner structure and complexity made them generally not easily interpretable.<n>In this work, a differentiable approximation of $L_0$ regularization is adapted into a logic-based neural network, the Multi-layer Logical Perceptron (MLLP), to study its efficacy in reducing the complexity of its discrete interpretable version, the Concept Rule Set (CRS)<n>The results are compared to alternatives like Random Binarization of the network weights, to determine if better results can be achieved when using a less-noisy technique that sparsifies the
  arXiv  Detail & Related papers  (2025-09-26T14:13:08Z)
• Optimizing Tensor Network Partitioning using Simulated Annealing [0.0]
  tensor networks have proven to be a valuable tool, for instance, in the classical simulation of (strongly correlated) quantum systems.<n>As the size of the systems increases, contracting larger tensor networks becomes computationally demanding.<n> Efficiently distributing the contraction task across multiple nodes is critical, as both computational and memory costs are highly sensitive to the chosen partitioning strategy.
  arXiv  Detail & Related papers  (2025-07-28T09:43:01Z)
• Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization [8.918295350787465]
  Current automatic parallel planning frameworks overlook the simultaneous consideration of node heterogeneity and dynamic network topology changes.<n>We introduce a strategy pruning technique to rapidly discard infeasible parallel configurations.<n>Preliminary evaluations confirm that our method notably enhances training performance on heterogeneous nodes.
  arXiv  Detail & Related papers  (2025-06-03T12:14:17Z)
• Efficient Reachability Analysis for Convolutional Neural Networks Using Hybrid Zonotopes [4.32258850473064]
  Existing set propagation-based reachability analysis methods for feedforward neural networks often struggle to achieve both scalability and accuracy. This work presents a novel set-based approach for computing the reachable sets of convolutional neural networks.
  arXiv  Detail & Related papers  (2025-03-13T19:45:26Z)
• Approximating Latent Manifolds in Neural Networks via Vanishing Ideals [20.464009622419766]
  We establish a connection between manifold learning and computational algebra by demonstrating how vanishing ideals can characterize the latent manifold of deep networks. We propose a new neural architecture that truncates a pretrained network at an intermediate layer, and approximates each class manifold via generators of the vanishing ideal. The resulting models have significantly fewer layers than their pretrained baselines, while maintaining comparable accuracy, achieving higher throughput and utilizing fewer parameters.
  arXiv  Detail & Related papers  (2025-02-20T21:23:02Z)
• SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning [49.83621156017321]
  SimBa is an architecture designed to scale up parameters in deep RL by injecting a simplicity bias.<n>By scaling up parameters with SimBa, the sample efficiency of various deep RL algorithms-including off-policy, on-policy, and unsupervised methods-is consistently improved.
  arXiv  Detail & Related papers  (2024-10-13T07:20:53Z)
• Hierarchical Multi-Marginal Optimal Transport for Network Alignment [52.206006379563306]
  Multi-network alignment is an essential prerequisite for joint learning on multiple networks. We propose a hierarchical multi-marginal optimal transport framework named HOT for multi-network alignment. Our proposed HOT achieves significant improvements over the state-of-the-art in both effectiveness and scalability.
  arXiv  Detail & Related papers  (2023-10-06T02:35:35Z)
• Generalization and Estimation Error Bounds for Model-based Neural Networks [78.88759757988761]
  We show that the generalization abilities of model-based networks for sparse recovery outperform those of regular ReLU networks. We derive practical design rules that allow to construct model-based networks with guaranteed high generalization.
  arXiv  Detail & Related papers  (2023-04-19T16:39:44Z)
• A pseudo-likelihood approach to community detection in weighted networks [4.111899441919165]
  We propose a pseudo-likelihood community estimation algorithm for networks with normally distributed edge weights. We prove that the estimates obtained by the proposed method are consistent under the assumption of homogeneous networks. We illustrate the method on simulated networks and on a fMRI dataset, where edge weights represent connectivity between brain regions.
  arXiv  Detail & Related papers  (2023-03-10T13:36:10Z)
• The Multiple Subnetwork Hypothesis: Enabling Multidomain Learning by Isolating Task-Specific Subnetworks in Feedforward Neural Networks [0.0]
  We identify a methodology and network representational structure which allows a pruned network to employ previously unused weights to learn subsequent tasks. We show that networks trained using our approaches are able to learn multiple tasks, which may be related or unrelated, in parallel or in sequence without sacrificing performance on any task or exhibiting catastrophic forgetting.
  arXiv  Detail & Related papers  (2022-07-18T15:07:13Z)
• i-SpaSP: Structured Neural Pruning via Sparse Signal Recovery [11.119895959906085]
  We propose a novel, structured pruning algorithm for neural networks -- the iterative, Sparse Structured Pruning, dubbed as i-SpaSP. i-SpaSP operates by identifying a larger set of important parameter groups within a network that contribute most to the residual between pruned and dense network output. It is shown to discover high-performing sub-networks and improve upon the pruning efficiency of provable baseline methodologies by several orders of magnitude.
  arXiv  Detail & Related papers  (2021-12-07T05:26:45Z)
• Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks [75.33431791218302]
  We study the training problem of deep neural networks and introduce an analytic approach to unveil hidden convexity in the optimization landscape. We consider a deep parallel ReLU network architecture, which also includes standard deep networks and ResNets as its special cases.
  arXiv  Detail & Related papers  (2021-10-18T18:00:36Z)
• Towards Understanding Theoretical Advantages of Complex-Reaction Networks [77.34726150561087]
  We show that a class of functions can be approximated by a complex-reaction network using the number of parameters. For empirical risk minimization, our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks.
  arXiv  Detail & Related papers  (2021-08-15T10:13:49Z)
• Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guarantee [30.86806523281873]
  Massive access is a critical design challenge of Internet of Things (IoT) networks. We consider the grant-free uplink transmission of an IoT network with a multiple-antenna base station (BS) and a large number of single-antenna IoT devices. We propose a novel algorithm unrolling framework based on the deep neural network to simultaneously achieve low computational complexity and high robustness.
  arXiv  Detail & Related papers  (2021-06-19T05:23:05Z)
• The Principles of Deep Learning Theory [19.33681537640272]
  This book develops an effective theory approach to understanding deep neural networks of practical relevance. We explain how these effectively-deep networks learn nontrivial representations from training. We show that the depth-to-width ratio governs the effective model complexity of the ensemble of trained networks.
  arXiv  Detail & Related papers  (2021-06-18T15:00:00Z)
• Reinforcement Learning for Adaptive Mesh Refinement [63.7867809197671]
  We propose a novel formulation of AMR as a Markov decision process and apply deep reinforcement learning to train refinement policies directly from simulation. The model sizes of these policy architectures are independent of the mesh size and hence scale to arbitrarily large and complex simulations.
  arXiv  Detail & Related papers  (2021-03-01T22:55:48Z)
• Distributed Optimization, Averaging via ADMM, and Network Topology [0.0]
  We study the connection between network topology and convergence rates for different algorithms on a real world problem of sensor localization. We also show interesting connections between ADMM and lifted Markov chains besides providing an explicitly characterization of its convergence.
  arXiv  Detail & Related papers  (2020-09-05T21:44:39Z)
• A Multi-Semantic Metapath Model for Large Scale Heterogeneous Network Representation Learning [52.83948119677194]
  We propose a multi-semantic metapath (MSM) model for large scale heterogeneous representation learning. Specifically, we generate multi-semantic metapath-based random walks to construct the heterogeneous neighborhood to handle the unbalanced distributions. We conduct systematical evaluations for the proposed framework on two challenging datasets: Amazon and Alibaba.
  arXiv  Detail & Related papers  (2020-07-19T22:50:20Z)
• ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
  We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks. Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
  arXiv  Detail & Related papers  (2020-06-28T23:09:27Z)
• Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems [59.804810122136345]
  We propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed. An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed. We show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.
  arXiv  Detail & Related papers  (2020-06-15T02:57:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.