Distributed Cooperative AI for Large-Scale Eigenvalue Computations Using Neural Networks

• URL: http://arxiv.org/abs/2409.06746v1
• Date: Tue, 10 Sep 2024 09:26:55 GMT
• Title: Distributed Cooperative AI for Large-Scale Eigenvalue Computations Using Neural Networks
• Authors: Ronald Katende,
• Abstract summary: This paper presents a novel method for eigenvalue computation using a distributed cooperative neural network framework. Our algorithm enables multiple autonomous agents to collaboratively estimate the smallest eigenvalue of large matrices.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel method for eigenvalue computation using a distributed cooperative neural network framework. Unlike traditional techniques that struggle with scalability in large systems, our decentralized algorithm enables multiple autonomous agents to collaboratively estimate the smallest eigenvalue of large matrices. Each agent uses a localized neural network model, refining its estimates through inter-agent communication. Our approach guarantees convergence to the true eigenvalue, even with communication failures or network disruptions. Theoretical analysis confirms the robustness and accuracy of the method, while empirical results demonstrate its better performance compared to some traditional centralized algorithms

Related papers

• Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizability [2.6708879445664584]
  This paper introduces a novel approach for assessing a newly trained model's performance based on another known model. The proposed method evaluates correlations by determining if, for each neuron in one network, there exists a neuron in the other network that produces similar output.
  arXiv  Detail & Related papers  (2024-08-15T22:57:39Z)
• LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks [52.46420522934253]
  We introduce LoRA-Ensemble, a parameter-efficient deep ensemble method for self-attention networks. By employing a single pre-trained self-attention network with weights shared across all members, we train member-specific low-rank matrices for the attention projections. Our method exhibits superior calibration compared to explicit ensembles and achieves similar or better accuracy across various prediction tasks and datasets.
  arXiv  Detail & Related papers  (2024-05-23T11:10:32Z)
• Using Cooperative Game Theory to Prune Neural Networks [7.3959659158152355]
  We show how solution concepts from cooperative game theory can be used to tackle the problem of pruning neural networks. We introduce a method called Game Theory Assisted Pruning (GTAP), which reduces the neural network's size while preserving its predictive accuracy.
  arXiv  Detail & Related papers  (2023-11-17T11:48:10Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection [10.191597755296163]
  We develop a tractable selection algorithm to efficiently identify the necessary feature combinations. Our proposed Sparse Interaction Additive Networks (SIAN) construct a bridge from simple and interpretable models to fully connected neural networks.
  arXiv  Detail & Related papers  (2022-09-19T19:57:17Z)
• Network Gradient Descent Algorithm for Decentralized Federated Learning [0.2867517731896504]
  We study a fully decentralized federated learning algorithm, which is a novel descent gradient algorithm executed on a communication-based network. In the NGD method, only statistics (e.g., parameter estimates) need to be communicated, minimizing the risk of privacy. We find that both the learning rate and the network structure play significant roles in determining the NGD estimator's statistical efficiency.
  arXiv  Detail & Related papers  (2022-05-06T02:53:31Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Learning Structures for Deep Neural Networks [99.8331363309895]
  We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience. We show that sparse coding can effectively maximize the entropy of the output signals. Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
  arXiv  Detail & Related papers  (2021-05-27T12:27:24Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Decentralized Deep Learning using Momentum-Accelerated Consensus [15.333413663982874]
  We consider the problem of decentralized deep learning where multiple agents collaborate to learn from a distributed dataset. We propose and analyze a novel decentralized deep learning algorithm where the agents interact over a fixed communication topology. Our algorithm is based on the heavy-ball acceleration method used in gradient-based protocol.
  arXiv  Detail & Related papers  (2020-10-21T17:39:52Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.