How deep the machine learning can be

• URL: http://arxiv.org/abs/2005.00872v1
• Date: Sat, 2 May 2020 16:06:31 GMT
• Title: How deep the machine learning can be
• Authors: J\'anos V\'egh
• Abstract summary: Machine learning is mostly based on the conventional computing (processors) This paper attempts to review some of the caveats, especially concerning scaling the computing performance of the AI solutions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Today we live in the age of artificial intelligence and machine learning; from small startups to HW or SW giants, everyone wants to build machine intelligence chips, applications. The task, however, is hard: not only because of the size of the problem: the technology one can utilize (and the paradigm it is based upon) strongly degrades the chances to succeed efficiently. Today the single-processor performance practically reached the limits the laws of nature enable. The only feasible way to achieve the needed high computing performance seems to be parallelizing many sequentially working units. The laws of the (massively) parallelized computing, however, are different from those experienced in connection with assembling and utilizing systems comprising just-a-few single processors. As machine learning is mostly based on the conventional computing (processors), we scrutinize the (known, but somewhat faded) laws of the parallel computing, concerning AI. This paper attempts to review some of the caveats, especially concerning scaling the computing performance of the AI solutions.

Related papers

• Benchmarking Predictive Coding Networks -- Made Simple [48.652114040426625]
  We first propose a library called PCX, whose focus lies on performance and simplicity. We use PCX to implement a large set of benchmarks for the community to use for their experiments.
  arXiv  Detail & Related papers  (2024-07-01T10:33:44Z)
• Thermodynamic Computing System for AI Applications [0.0]
  Physics-based hardware, such as thermodynamic computing, has the potential to provide a fast, low-power means to accelerate AI primitives. We present the first continuous-variable thermodynamic computer, which we call the processing unit (SPU)
  arXiv  Detail & Related papers  (2023-12-08T05:22:04Z)
• Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
  Predictive coding (PC) has shown promising performance in machine intelligence tasks. PC can model information processing in different brain areas, can be used in cognitive control and robotics.
  arXiv  Detail & Related papers  (2023-08-15T16:37:16Z)
• Reliable AI: Does the Next Generation Require Quantum Computing? [71.84486326350338]
  We show that digital hardware is inherently constrained in solving problems about optimization, deep learning, or differential equations. In contrast, analog computing models, such as the Blum-Shub-Smale machine, exhibit the potential to surmount these limitations.
  arXiv  Detail & Related papers  (2023-07-03T19:10:45Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)
• Divide & Conquer Imitation Learning [75.31752559017978]
  Imitation Learning can be a powerful approach to bootstrap the learning process. We present a novel algorithm designed to imitate complex robotic tasks from the states of an expert trajectory. We show that our method imitates a non-holonomic navigation task and scales to a complex simulated robotic manipulation task with very high sample efficiency.
  arXiv  Detail & Related papers  (2022-04-15T09:56:50Z)
• End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking [52.05847268235338]
  We show how machine learning systems can perform logical extrapolation without overthinking problems. We propose a recall architecture that keeps an explicit copy of the problem instance in memory so that it cannot be forgotten. We also employ a progressive training routine that prevents the model from learning behaviors that are specific to number and instead pushes it to learn behaviors that can be repeated indefinitely.
  arXiv  Detail & Related papers  (2022-02-11T18:43:28Z)
• Artificial Intelligence in the Low-Level Realm -- A Survey [0.0]
  We seek methods and efforts that exploit AI approaches, specifically machine learning, in the OSes' primary responsibilities. In other words, the main question to be answered is how AI has played/can play a role directly in improving the traditional OS kernel main tasks.
  arXiv  Detail & Related papers  (2021-09-19T19:36:54Z)
• Hybrid Intelligence [4.508830262248694]
  We argue that the most likely paradigm for the division of labor between humans and machines in the next decades is Hybrid Intelligence. This concept aims at using the complementary strengths of human intelligence and AI, so that they can perform better than each of the two could separately.
  arXiv  Detail & Related papers  (2021-05-03T08:56:09Z)
• Which scaling rule applies to Artificial Neural Networks [0.0]
  We show that cooperating and communicating computing systems, comprising segregated single processors, have severe performance limitations. The paper starts from von Neumann's original model, without neglecting the transfer time apart from processing time; derives an appropriate interpretation and handling for Amdahl's law.
  arXiv  Detail & Related papers  (2020-05-15T19:52:55Z)
• SOL: Effortless Device Support for AI Frameworks without Source Code Changes [1.030051577369649]
  We introduce SOL, an AI acceleration that provides a hardware abstraction layer that allows us to transparently support heterogeneous hardware. As a proof of concept, we implemented SOL for PyTorch with three backends: CPU, GPU and vector processors.
  arXiv  Detail & Related papers  (2020-03-24T07:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.