An Open-Source HW-SW Co-Development Framework Enabling Efficient Multi-Accelerator Systems

• URL: http://arxiv.org/abs/2508.14582v1
• Date: Wed, 20 Aug 2025 10:04:21 GMT
• Title: An Open-Source HW-SW Co-Development Framework Enabling Efficient Multi-Accelerator Systems
• Abstract summary: Heterogeneous accelerator-centric compute clusters are emerging as efficient solutions for diverse AI workloads.<n>We present SNAX, an open-source integrated HW-SW framework enabling efficient multi-accelerator platforms.
• Score: 2.869405001718283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Heterogeneous accelerator-centric compute clusters are emerging as efficient solutions for diverse AI workloads. However, current integration strategies often compromise data movement efficiency and encounter compatibility issues in hardware and software. This prevents a unified approach that balances performance and ease of use. To this end, we present SNAX, an open-source integrated HW-SW framework enabling efficient multi-accelerator platforms through a novel hybrid-coupling scheme, consisting of loosely coupled asynchronous control and tightly coupled data access. SNAX brings reusable hardware modules designed to enhance compute accelerator utilization, and its customizable MLIR-based compiler to automate key system management tasks, jointly enabling rapid development and deployment of customized multi-accelerator compute clusters. Through extensive experimentation, we demonstrate SNAX's efficiency and flexibility in a low-power heterogeneous SoC. Accelerators can easily be integrated and programmed to achieve > 10x improvement in neural network performance compared to other accelerator systems while maintaining accelerator utilization of > 90% in full system operation.

Related papers

• Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks [48.421359451414155]
  Load imbalance across edge cloud and layers degrades latency performance in.<n> hierarchical edge-cloud computing systems under dynamic task arrivals and heterogeneous resources.<n>We propose a two-time deep learning study with an action space (2MDRL-LA) to jointly optimize service placement.<n>It achieves a 20.8% reduction in average e2e latency and a 13% improvement in resource utilization over the scheme.
  arXiv  Detail & Related papers  (2026-06-30T11:56:15Z)
• Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation [10.567064384827196]
  Complex-Valued Neural Networks (CVNNs) have advantages in handling tasks that involve complex numbers.<n>Existing CVNNs are unable to quantify predictive uncertainty.<n>We propose, for the first time, dropout-based Bayesian Complex-Valued Neural Networks (BayesCVNNs) to enable uncertainty quantification.
  arXiv  Detail & Related papers  (2026-04-21T21:06:00Z)
• MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs [45.487954219664026]
  MATCHA generates highly concurrent schedules for parallel, heterogeneous accelerators.<n>On the Tinyerf benchmark, MATCHA improves accelerator utilization and reduces latency by up to 35%.
  arXiv  Detail & Related papers  (2026-04-10T09:04:34Z)
• AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization [84.25316984309725]
  Integration of dynamic, sparse structures with parameter-efficient adapters (e.g., LoRA) is a powerful technique for enhancing Large Language Models (LLMs)<n>Despite minimal increases in computational load, the computation latency often skyrockets, leading to decoding speeds slowing by over 2.5 times.<n>AdaFuse is a framework built on a tight co-design between the algorithm and the underlying hardware system to enable efficient dynamic adapter execution.
  arXiv  Detail & Related papers  (2026-03-12T12:46:42Z)
• Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs [61.953548065938385]
  We introduce the ''Three Taxes'' (Bulk Synchronous, Inter- Kernel Data Locality, and Kernel Launch Overhead) as an analytical framework.<n>We propose moving beyond the rigid BSP model to address key inefficiencies in distributed GPU execution.<n>We observe a 10-20% speedup in end-to-end latency over BSP-based approaches.
  arXiv  Detail & Related papers  (2025-11-04T01:15:44Z)
• CollaPipe: Adaptive Segment-Optimized Pipeline Parallelism for Collaborative LLM Training in Heterogeneous Edge Networks [57.95170323315603]
  We introduce CollaPipe, a distributed learning framework that integrates collaborative pipeline parallelism with federated aggregation to support self-evolving networks.<n>In CollaPipe, the encoder part is adaptively partitioned into variable-sized segments and deployed across mobile devices for pipeline-parallel training, while the decoder is deployed on edge servers to handle generative tasks.<n>To enhance training efficiency, we formulate a joint optimization problem that adaptively allocates model segments, micro-batches, bandwidth, and transmission power.
  arXiv  Detail & Related papers  (2025-09-24T07:54:01Z)
• Joint Channel Estimation and Computation Offloading in Fluid Antenna-assisted MEC Networks [81.36647816787713]
  We propose an FA-assisted offloading framework to minimize the delay of channel estimation.<n>We show that the proposed system significantly reduces the accuracy under efficient communication.
  arXiv  Detail & Related papers  (2025-09-16T08:48:44Z)
• KAITIAN: A Unified Communication Framework for Enabling Efficient Collaboration Across Heterogeneous Accelerators in Embodied AI Systems [5.241889216655924]
  KAITIAN is a novel distributed communication framework for AI workloads.<n>It integrates vendor-optimized communication libraries for intra-group efficiency with general-purpose communication protocols for inter-group interoperability.<n>It can accelerate training time by up to 42% compared to baseline homogeneous systems.
  arXiv  Detail & Related papers  (2025-05-15T11:29:43Z)
• Adaptive Orchestration for Large-Scale Inference on Heterogeneous Accelerator Systems Balancing Cost, Performance, and Resilience [0.46040036610482665]
  This paper proposes a hardware-agnostic control loop that adaptively allocates requests across heterogeneous accelerators.<n>The framework consistently meets latency targets, automatically redirects traffic during capacity shortfalls, and capitalizes on lower-cost accelerators.
  arXiv  Detail & Related papers  (2025-03-25T21:20:11Z)
• Exploring the Potential of Wireless-enabled Multi-Chip AI Accelerators [2.2305608711864555]
  We show that wireless interconnects can lead to speedups of 10% on average and 20% maximum.<n>We highlight the importance of load balancing between the wired and wireless interconnects.
  arXiv  Detail & Related papers  (2025-01-29T11:00:09Z)
• Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks [60.085771314013044]
  Low-altitude economy holds significant potential for development in areas such as communication and sensing.<n>We propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN.
  arXiv  Detail & Related papers  (2024-12-14T06:17:33Z)
• OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling [2.869405001718283]
  Deep neural networks (DNNs) face significant challenges when deployed on resource-constrained extreme edge devices. We propose OpenGeMM, an open-source acceleration platform, jointly demonstrating high efficiency and utilization.
  arXiv  Detail & Related papers  (2024-11-14T15:58:46Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)
• Communication-Control Codesign for Large-Scale Wireless Networked Control Systems [80.30532872347668]
  Wireless Networked Control Systems (WNCSs) are essential to Industry 4.0, enabling flexible control in applications, such as drone swarms and autonomous robots. We propose a practical WNCS model that captures correlated dynamics among multiple control loops with spatially distributed sensors and actuators sharing limited wireless resources over multi-state Markov block-fading channels. We develop a Deep Reinforcement Learning (DRL) algorithm that efficiently handles the hybrid action space, captures communication-control correlations, and ensures robust training despite sparse cross-domain variables and floating control inputs.
  arXiv  Detail & Related papers  (2024-10-15T06:28:21Z)
• MARS: Exploiting Multi-Level Parallelism for DNN Workloads on Adaptive Multi-Accelerator Systems [27.490645446510033]
  We propose a novel mapping framework that can perform computation-aware accelerator selection and apply communication-aware sharding strategies to maximize parallelism. We show that MARS can achieve 32.2% latency reduction on average for typical DNN workloads compared to the baseline, and 59.4% latency reduction on heterogeneous models compared to the corresponding state-of-the-art method.
  arXiv  Detail & Related papers  (2023-07-23T05:50:37Z)
• Reconfigurable Distributed FPGA Cluster Design for Deep Learning Accelerators [59.11160990637615]
  We propose a distributed system based on lowpower embedded FPGAs designed for edge computing applications. The proposed system can simultaneously execute diverse Neural Network (NN) models, arrange the graph in a pipeline structure, and manually allocate greater resources to the most computationally intensive layers of the NN graph.
  arXiv  Detail & Related papers  (2023-05-24T16:08:55Z)
• On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solver [59.13397937903832]
  We introduce a deep learning-based corrector called Neural Vector (NeurVec) NeurVec can compensate for integration errors and enable larger time step sizes in simulations. Our experiments on a variety of complex dynamical system benchmarks demonstrate that NeurVec exhibits remarkable generalization capability.
  arXiv  Detail & Related papers  (2022-08-07T09:02:18Z)
• FPGA-based AI Smart NICs for Scalable Distributed AI Training Systems [62.20308752994373]
  We propose a new smart network interface card (NIC) for distributed AI training systems using field-programmable gate arrays (FPGAs) Our proposed FPGA-based AI smart NIC enhances overall training performance by 1.6x at 6 nodes, with an estimated 2.5x performance improvement at 32 nodes, compared to the baseline system using conventional NICs.
  arXiv  Detail & Related papers  (2022-04-22T21:57:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.