The Grand Illusion: The Myth of Software Portability and Implications
for ML Progress
- URL: http://arxiv.org/abs/2309.07181v1
- Date: Tue, 12 Sep 2023 22:11:55 GMT
- Title: The Grand Illusion: The Myth of Software Portability and Implications
for ML Progress
- Authors: Fraser Mince, Dzung Dinh, Jonas Kgomo, Neil Thompson, Sara Hooker
- Abstract summary: We conduct a large-scale study of the portability of mainstream ML frameworks across different hardware types.
We find that frameworks can lose more than 40% of their key functions when ported to other hardware.
Our results suggest that specialization of hardware impedes innovation in machine learning research.
- Score: 4.855502010124377
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Pushing the boundaries of machine learning often requires exploring different
hardware and software combinations. However, the freedom to experiment across
different tooling stacks can be at odds with the drive for efficiency, which
has produced increasingly specialized AI hardware and incentivized
consolidation around a narrow set of ML frameworks. Exploratory research can be
restricted if software and hardware are co-evolving, making it even harder to
stray away from mainstream ideas that work well with popular tooling stacks.
While this friction increasingly impacts the rate of innovation in machine
learning, to our knowledge the lack of portability in tooling has not been
quantified. In this work, we ask: How portable are popular ML software
frameworks? We conduct a large-scale study of the portability of mainstream ML
frameworks across different hardware types. Our findings paint an uncomfortable
picture -- frameworks can lose more than 40% of their key functions when ported
to other hardware. Worse, even when functions are portable, the slowdown in
their performance can be extreme and render performance untenable.
Collectively, our results reveal how costly straying from a narrow set of
hardware-software combinations can be - and suggest that specialization of
hardware impedes innovation in machine learning research.
Related papers
- Multi-objective Differentiable Neural Architecture Search [58.67218773054753]
We propose a novel NAS algorithm that encodes user preferences for the trade-off between performance and hardware metrics.
Our method outperforms existing MOO NAS methods across a broad range of qualitatively different search spaces and datasets.
arXiv Detail & Related papers (2024-02-28T10:09:04Z) - MLXP: A Framework for Conducting Replicable Experiments in Python [63.37350735954699]
We propose MLXP, an open-source, simple, and lightweight experiment management tool based on Python.
It streamlines the experimental process with minimal overhead while ensuring a high level of practitioner overhead.
arXiv Detail & Related papers (2024-02-21T14:22:20Z) - Random resistive memory-based deep extreme point learning machine for
unified visual processing [67.51600474104171]
We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM)
Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
arXiv Detail & Related papers (2023-12-14T09:46:16Z) - On The Fairness Impacts of Hardware Selection in Machine Learning [50.03224106965757]
This paper investigates the influence of hardware on the delicate balance between model performance and fairness.
We demonstrate that hardware choices can exacerbate existing disparities, attributing these discrepancies to variations in gradient flows and loss surfaces across different demographic groups.
arXiv Detail & Related papers (2023-12-06T20:24:17Z) - FusionAI: Decentralized Training and Deploying LLMs with Massive
Consumer-Level GPUs [57.12856172329322]
We envision a decentralized system unlocking the potential vast untapped consumer-level GPU.
This system faces critical challenges, including limited CPU and GPU memory, low network bandwidth, the variability of peer and device heterogeneity.
arXiv Detail & Related papers (2023-09-03T13:27:56Z) - A review of TinyML [0.0]
The TinyML concept for embedded machine learning attempts to push such diversity from usual high-end approaches to low-end applications.
TinyML is a rapidly expanding interdisciplinary topic at the convergence of machine learning, software, and hardware.
This paper explores how TinyML can benefit a few specific industrial fields, its obstacles, and its future scope.
arXiv Detail & Related papers (2022-11-05T06:02:08Z) - Flashlight: Enabling Innovation in Tools for Machine Learning [50.63188263773778]
We introduce Flashlight, an open-source library built to spur innovation in machine learning tools and systems.
We see Flashlight as a tool enabling research that can benefit widely used libraries downstream and bring machine learning and systems researchers closer together.
arXiv Detail & Related papers (2022-01-29T01:03:29Z) - Learned Hardware/Software Co-Design of Neural Accelerators [20.929918108940093]
Deep learning software stacks and hardware accelerators are diverse and vast.
Prior work considers software optimizations separately from hardware architectures, effectively reducing the search space.
This paper casts the problem as hardware/software co-design, with the goal of automatically identifying desirable points in the joint design space.
arXiv Detail & Related papers (2020-10-05T15:12:52Z) - MLOS: An Infrastructure for Automated Software Performance Engineering [14.244308246225744]
We present MLOS, an ML-powered infrastructure and methodology to democratize Software Performance Engineering.
MLOS enables continuous, instance-level, robust, and trackable systems optimization.
We are in the process of open-sourcing the MLOS core infrastructure, and we are engaging with academic institutions to create an educational program around Software 2.0 and MLOS ideas.
arXiv Detail & Related papers (2020-06-01T22:38:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.