A Survey From Distributed Machine Learning to Distributed Deep Learning

• URL: http://arxiv.org/abs/2307.05232v2
• Date: Sat, 9 Sep 2023 12:17:05 GMT
• Title: A Survey From Distributed Machine Learning to Distributed Deep Learning
• Authors: Mohammad Dehghani, Zahra Yazdanparast
• Abstract summary: Distributed machine learning has been proposed, which involves distributing the data and algorithm across several machines. We divide these algorithms in classification and clustering (traditional machine learning), deep learning and deep reinforcement learning groups. Based on the investigation of the mentioned algorithms, we highlighted the limitations that should be addressed in future research.
• Score: 0.356008609689971
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Artificial intelligence has made remarkable progress in handling complex tasks, thanks to advances in hardware acceleration and machine learning algorithms. However, to acquire more accurate outcomes and solve more complex issues, algorithms should be trained with more data. Processing this huge amount of data could be time-consuming and require a great deal of computation. To address these issues, distributed machine learning has been proposed, which involves distributing the data and algorithm across several machines. There has been considerable effort put into developing distributed machine learning algorithms, and different methods have been proposed so far. We divide these algorithms in classification and clustering (traditional machine learning), deep learning and deep reinforcement learning groups. Distributed deep learning has gained more attention in recent years and most of the studies have focused on this approach. Therefore, we mostly concentrate on this category. Based on the investigation of the mentioned algorithms, we highlighted the limitations that should be addressed in future research.

Related papers

• From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models [63.188607839223046]
  This survey focuses on the benefits of scaling compute during inference. We explore three areas under a unified mathematical formalism: token-level generation algorithms, meta-generation algorithms, and efficient generation.
  arXiv  Detail & Related papers  (2024-06-24T17:45:59Z)
• Machine Learning Augmented Branch and Bound for Mixed Integer Linear Programming [11.293025183996832]
  Mixed Linear Programming (MILP) offers a powerful modeling language for a wide range of applications. In recent years, there has been an explosive development in the use of machine learning algorithms for enhancing all main tasks involved in the branch-and-bound algorithm. In particular, we give detailed attention to machine learning algorithms that automatically optimize some metric of branch-and-bound efficiency.
  arXiv  Detail & Related papers  (2024-02-08T09:19:26Z)
• Accelerating Machine Learning Algorithms with Adaptive Sampling [1.539582851341637]
  It is often sufficient to substitute computationally intensive subroutines with a special kind of randomized counterparts that results in almost no degradation in quality. This thesis demonstrates that it is often sufficient, instead, to substitute computationally intensive subroutines with a special kind of randomized counterparts that results in almost no degradation in quality.
  arXiv  Detail & Related papers  (2023-09-25T15:25:59Z)
• MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge [110.7678032481059]
  We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. Our results show that current machine learning search algorithms may already be sensitive enough in limited parameter regions to be useful for some production settings.
  arXiv  Detail & Related papers  (2022-09-22T16:44:59Z)
• Research Trends and Applications of Data Augmentation Algorithms [77.34726150561087]
  We identify the main areas of application of data augmentation algorithms, the types of algorithms used, significant research trends, their progression over time and research gaps in data augmentation literature. We expect readers to understand the potential of data augmentation, as well as identify future research directions and open questions within data augmentation research.
  arXiv  Detail & Related papers  (2022-07-18T11:38:32Z)
• Benchmarking Processor Performance by Multi-Threaded Machine Learning Algorithms [0.0]
  In this paper, I will make a performance comparison of multi-threaded machine learning clustering algorithms. I will be working on Linear Regression, Random Forest, and K-Nearest Neighbors to determine the performance characteristics of the algorithms.
  arXiv  Detail & Related papers  (2021-09-11T13:26:58Z)
• Launchpad: A Programming Model for Distributed Machine Learning Research [12.618286123360233]
  We introduce Launchpad, a programming model that simplifies the process of defining and launching distributed systems. We describe our framework, its design philosophy and implementation, and give a number of examples of common learning algorithms whose designs are greatly simplified by this approach.
  arXiv  Detail & Related papers  (2021-06-07T17:02:10Z)
• Ten Quick Tips for Deep Learning in Biology [116.78436313026478]
  Machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling. Deep learning has become its own subfield of machine learning. In the context of biological research, deep learning has been increasingly used to derive novel insights from high-dimensional biological data.
  arXiv  Detail & Related papers  (2021-05-29T21:02:44Z)
• A Survey on Large-scale Machine Learning [67.6997613600942]
  Machine learning can provide deep insights into data, allowing machines to make high-quality predictions. Most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. Large-scale Machine Learning aims to learn patterns from big data with comparable performance efficiently.
  arXiv  Detail & Related papers  (2020-08-10T06:07:52Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)
• Guidelines for enhancing data locality in selected machine learning algorithms [0.0]
  We analyze one of the means to increase the performances of machine learning algorithms which is exploiting data locality. Repeated data access can be seen as redundancy in data movement. This work also identifies some of the opportunities for avoiding these redundancies by directly reusing results.
  arXiv  Detail & Related papers  (2020-01-09T14:16:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.