Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning

• URL: http://arxiv.org/abs/2302.07946v3
• Date: Wed, 18 Oct 2023 08:35:00 GMT
• Title: Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning
• Authors: Gianluca Mittone, Nicol\`o Tonci, Robert Birke, Iacopo Colonnelli, Doriana Medi\'c, Andrea Bartolini, Roberto Esposito, Emanuele Parisi, Francesco Beneventi, Mirko Polato, Massimo Torquati, Luca Benini, Marco Aldinucci
• Abstract summary: Decentralised Machine Learning (DML) enables collaborative machine learning without centralised input data. We map DML schemes to an underlying parallel programming library. We experiment with it by generating different working DML schemes on x86-64 and ARM platforms and an emerging RISC-V one. As a byproduct, we introduce a RISC-V porting of the PyTorch framework, the first publicly available to our knowledge.
• Score: 12.18598759507803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decentralised Machine Learning (DML) enables collaborative machine learning without centralised input data. Federated Learning (FL) and Edge Inference are examples of DML. While tools for DML (especially FL) are starting to flourish, many are not flexible and portable enough to experiment with novel processors (e.g., RISC-V), non-fully connected network topologies, and asynchronous collaboration schemes. We overcome these limitations via a domain-specific language allowing us to map DML schemes to an underlying middleware, i.e. the FastFlow parallel programming library. We experiment with it by generating different working DML schemes on x86-64 and ARM platforms and an emerging RISC-V one. We characterise the performance and energy efficiency of the presented schemes and systems. As a byproduct, we introduce a RISC-V porting of the PyTorch framework, the first publicly available to our knowledge.

Related papers

• TDML -- A Trustworthy Distributed Machine Learning Framework [7.302091381583343]
  The rapid advancement of large models (LM) has intensified the demand for computing resources. This demand is exacerbated by limited availability due to supply chain delays and monopolistic acquisition by major tech firms. We propose a textittrustworthy distributed machine learning (TDML) framework that leverages guidance to coordinate remote trainers and validate workloads.
  arXiv  Detail & Related papers  (2024-07-10T03:22:28Z)
• Distributed Inference and Fine-tuning of Large Language Models Over The Internet [91.00270820533272]
  Large language models (LLMs) are useful in many NLP tasks and become more capable with size. These models require high-end hardware, making them inaccessible to most researchers. We develop fault-tolerant inference algorithms and load-balancing protocols that automatically assign devices to maximize the total system throughput.
  arXiv  Detail & Related papers  (2023-12-13T18:52:49Z)
• RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs [32.01139974519813]
  We present RedCoast, a tool crafted to automate distributed training and inference for large language models (LLMs) We also propose a mechanism that allows for the customization of diverse ML pipelines through the definition of merely three functions. As a result, Redco implementations exhibit significantly fewer lines of code compared to their official counterparts.
  arXiv  Detail & Related papers  (2023-10-25T04:32:35Z)
• In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD [51.04126395480625]
  Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. This work offers a solution by simplifying this coupling and enabling in situ training and inference on heterogeneous clusters.
  arXiv  Detail & Related papers  (2023-06-22T14:07:54Z)
• Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networks [49.19502459827366]
  Federated learning (FL) has been promoted as a popular technique for training machine learning (ML) models over edge/fog networks. Traditional implementations of FL have largely neglected the potential for inter-network cooperation. We advocate for cooperative federated learning (CFL), a cooperative edge/fog ML paradigm built on device-to-device (D2D) and device-to-server (D2S) interactions.
  arXiv  Detail & Related papers  (2023-03-15T04:41:36Z)
• MDE for Machine Learning-Enabled Software Systems: A Case Study and Comparison of MontiAnna & ML-Quadrat [5.839906946900443]
  We propose to adopt the MDE paradigm for the development of Machine Learning-enabled software systems with a focus on the Internet of Things (IoT) domain. We illustrate how two state-of-the-art open-source modeling tools, namely MontiAnna and ML-Quadrat can be used for this purpose as demonstrated through a case study.
  arXiv  Detail & Related papers  (2022-09-15T13:21:16Z)
• Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning [55.198301429316125]
  Machine learning (ML) is a key technique for big-data-driven modelling and analysis of massive Internet of Things (IoT) based intelligent and ubiquitous computing. For fast-increasing applications and data amounts, distributed learning is a promising emerging paradigm since it is often impractical or inefficient to share/aggregate data. This paper studies the problem of training an ML model over decentralized systems, where data are distributed over many user devices.
  arXiv  Detail & Related papers  (2022-02-07T15:04:15Z)
• Enabling Un-/Semi-Supervised Machine Learning for MDSE of the Real-World CPS/IoT Applications [0.5156484100374059]
  We propose a novel approach to support domain-specific Model-Driven Software Engineering (MDSE) for the real-world use-case scenarios of smart Cyber-Physical Systems (CPS) and the Internet of Things (IoT) We argue that the majority of available data in the nature for Artificial Intelligence (AI) are unlabeled. Hence, unsupervised and/or semi-supervised ML approaches are the practical choices. Our proposed approach is fully implemented and integrated with an existing state-of-the-art MDSE tool to serve the CPS/IoT domain.
  arXiv  Detail & Related papers  (2021-07-06T15:51:39Z)
• FedML: A Research Library and Benchmark for Federated Machine Learning [55.09054608875831]
  Federated learning (FL) is a rapidly growing research field in machine learning. Existing FL libraries cannot adequately support diverse algorithmic development. We introduce FedML, an open research library and benchmark to facilitate FL algorithm development and fair performance comparison.
  arXiv  Detail & Related papers  (2020-07-27T13:02:08Z)
• Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G Networks [84.2155885234293]
  We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC. To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
  arXiv  Detail & Related papers  (2020-02-22T14:38:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.