Single-GPU GNN Systems: Traps and Pitfalls

• URL: http://arxiv.org/abs/2402.03548v1
• Date: Mon, 5 Feb 2024 22:07:58 GMT
• Title: Single-GPU GNN Systems: Traps and Pitfalls
• Authors: Yidong Gong, Arnab Tarafder, Saima Afrin, and Pradeep Kumar
• Abstract summary: In-depth analysis shows that it leads to a chain of pitfalls in the system design and evaluation process. New reference system is developed to establish a new line of optimizations rooted in solving the system-design pitfalls efficiently and practically.
• Score: 1.5669062603298305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The current graph neural network (GNN) systems have established a clear trend of not showing training accuracy results, and directly or indirectly relying on smaller datasets for evaluations majorly. Our in-depth analysis shows that it leads to a chain of pitfalls in the system design and evaluation process, questioning the practicality of many of the proposed system optimizations, and affecting conclusions and lessons learned. We analyze many single-GPU systems and show the fundamental impact of these pitfalls. We further develop hypotheses, recommendations, and evaluation methodologies, and provide future directions. Finally, a new reference system is developed to establish a new line of optimizations rooted in solving the system-design pitfalls efficiently and practically. The proposed design can productively be integrated into prior works, thereby truly advancing the state-of-the-art.

Related papers

• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• End-to-End Learning for Stochastic Optimization: A Bayesian Perspective [9.356870107137093]
  We develop a principled approach to end-to-end learning in optimization. We show that the standard end-to-end learning algorithm admits a Bayesian interpretation and trains a posterior Bayes action map. We then propose new end-to-end learning algorithms for training decision maps.
  arXiv  Detail & Related papers  (2023-06-07T05:55:45Z)
• Learning Large-scale Neural Fields via Context Pruned Meta-Learning [60.93679437452872]
  We introduce an efficient optimization-based meta-learning technique for large-scale neural field training. We show how gradient re-scaling at meta-test time allows the learning of extremely high-quality neural fields. Our framework is model-agnostic, intuitive, straightforward to implement, and shows significant reconstruction improvements for a wide range of signals.
  arXiv  Detail & Related papers  (2023-02-01T17:32:16Z)
• Analysis and Design of Quadratic Neural Networks for Regression, Classification, and Lyapunov Control of Dynamical Systems [0.0]
  This paper addresses the analysis and design of quadratic neural networks. Networks offer several advantages, the most important of which are the fact that the architecture is a by-product of the design and is not determined a-priori. Several examples will show the effectiveness of quadratic neural networks in applications.
  arXiv  Detail & Related papers  (2022-07-26T18:10:05Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control [2.2725929250900947]
  We tailor the projected gradient descent (PGD) method developed in the adversarial learning community as a general-purpose ROA analysis tool. We show that the ROA analysis can be approximated as a constrained problem whose goal is to find the worst-case initial condition. We present two PGD-based iterative methods which can be used to solve the resultant constrained problem.
  arXiv  Detail & Related papers  (2022-01-03T18:46:58Z)
• Supervised DKRC with Images for Offline System Identification [77.34726150561087]
  Modern dynamical systems are becoming increasingly non-linear and complex. There is a need for a framework to model these systems in a compact and comprehensive representation for prediction and control. Our approach learns these basis functions using a supervised learning approach.
  arXiv  Detail & Related papers  (2021-09-06T04:39:06Z)
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions [80.78077900288868]
  We decompose the design methodology of early-exit networks to its key components and survey the recent advances in each one of them. We position early-exiting against other efficient inference solutions and provide our insights on the current challenges and most promising future directions for research in the field.
  arXiv  Detail & Related papers  (2021-06-09T12:33:02Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization [17.25824905485415]
  We present Neural-iLQR, a learning-aided shooting method over the unconstrained control space. It is shown to outperform the conventional iLQR significantly in the presence of inaccuracies in system models.
  arXiv  Detail & Related papers  (2020-11-21T07:17:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.