NeuroBench: A Framework for Benchmarking Neuromorphic Computing
Algorithms and Systems
- URL: http://arxiv.org/abs/2304.04640v3
- Date: Wed, 17 Jan 2024 20:40:28 GMT
- Title: NeuroBench: A Framework for Benchmarking Neuromorphic Computing
Algorithms and Systems
- Authors: Jason Yik, Korneel Van den Berghe, Douwe den Blanken, Younes
Bouhadjar, Maxime Fabre, Paul Hueber, Denis Kleyko, Noah Pacik-Nelson,
Pao-Sheng Vincent Sun, Guangzhi Tang, Shenqi Wang, Biyan Zhou, Soikat Hasan
Ahmed, George Vathakkattil Joseph, Benedetto Leto, Aurora Micheli, Anurag
Kumar Mishra, Gregor Lenz, Tao Sun, Zergham Ahmed, Mahmoud Akl, Brian
Anderson, Andreas G. Andreou, Chiara Bartolozzi, Arindam Basu, Petrut Bogdan,
Sander Bohte, Sonia Buckley, Gert Cauwenberghs, Elisabetta Chicca, Federico
Corradi, Guido de Croon, Andreea Danielescu, Anurag Daram, Mike Davies, Yigit
Demirag, Jason Eshraghian, Tobias Fischer, Jeremy Forest, Vittorio Fra, Steve
Furber, P. Michael Furlong, William Gilpin, Aditya Gilra, Hector A. Gonzalez,
Giacomo Indiveri, Siddharth Joshi, Vedant Karia, Lyes Khacef, James C.
Knight, Laura Kriener, Rajkumar Kubendran, Dhireesha Kudithipudi, Yao-Hong
Liu, Shih-Chii Liu, Haoyuan Ma, Rajit Manohar, Josep Maria Margarit-Taul\'e,
Christian Mayr, Konstantinos Michmizos, Dylan Muir, Emre Neftci, Thomas
Nowotny, Fabrizio Ottati, Ayca Ozcelikkale, Priyadarshini Panda, Jongkil
Park, Melika Payvand, Christian Pehle, Mihai A. Petrovici, Alessandro Pierro,
Christoph Posch, Alpha Renner, Yulia Sandamirskaya, Clemens JS Schaefer,
Andr\'e van Schaik, Johannes Schemmel, Samuel Schmidgall, Catherine Schuman,
Jae-sun Seo, Sadique Sheik, Sumit Bam Shrestha, Manolis Sifalakis, Amos
Sironi, Matthew Stewart, Kenneth Stewart, Terrence C. Stewart, Philipp
Stratmann, Jonathan Timcheck, Nergis T\"omen, Gianvito Urgese, Marian
Verhelst, Craig M. Vineyard, Bernhard Vogginger, Amirreza Yousefzadeh, Fatima
Tuz Zohora, Charlotte Frenkel, Vijay Janapa Reddi
- Abstract summary: NeuroBench is a benchmark framework for neuromorphic computing algorithms and systems.
NeuroBench is a collaboratively-designed effort from an open community of nearly 100 co-authors across over 50 institutions in industry and academia.
- Score: 51.8066436083197
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Neuromorphic computing shows promise for advancing computing efficiency and
capabilities of AI applications using brain-inspired principles. However, the
neuromorphic research field currently lacks standardized benchmarks, making it
difficult to accurately measure technological advancements, compare performance
with conventional methods, and identify promising future research directions.
Prior neuromorphic computing benchmark efforts have not seen widespread
adoption due to a lack of inclusive, actionable, and iterative benchmark design
and guidelines. To address these shortcomings, we present NeuroBench: a
benchmark framework for neuromorphic computing algorithms and systems.
NeuroBench is a collaboratively-designed effort from an open community of
nearly 100 co-authors across over 50 institutions in industry and academia,
aiming to provide a representative structure for standardizing the evaluation
of neuromorphic approaches. The NeuroBench framework introduces a common set of
tools and systematic methodology for inclusive benchmark measurement,
delivering an objective reference framework for quantifying neuromorphic
approaches in both hardware-independent (algorithm track) and
hardware-dependent (system track) settings. In this article, we present initial
performance baselines across various model architectures on the algorithm track
and outline the system track benchmark tasks and guidelines. NeuroBench is
intended to continually expand its benchmarks and features to foster and track
the progress made by the research community.
Related papers
- Genetic Motifs as a Blueprint for Mismatch-Tolerant Neuromorphic Computing [1.8292454465322363]
Mixed-signal implementations of SNNs offer a promising solution to edge computing applications.
Device mismatch in the analog circuits of these neuromorphic processors poses a significant challenge to the deployment of robust processing.
We introduce a novel architectural solution inspired by biological development to address this issue.
arXiv Detail & Related papers (2024-10-25T09:04:50Z) - A Realistic Simulation Framework for Analog/Digital Neuromorphic Architectures [73.65190161312555]
ARCANA is a spiking neural network simulator designed to account for the properties of mixed-signal neuromorphic circuits.
We show how the results obtained provide a reliable estimate of the behavior of the spiking neural network trained in software.
arXiv Detail & Related papers (2024-09-23T11:16:46Z) - GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models.
GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies.
We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
arXiv Detail & Related papers (2024-06-01T08:01:05Z) - Reachability Analysis of a General Class of Neural Ordinary Differential
Equations [7.774796410415387]
Continuous deep learning models, referred to as Neural Ordinary Differential Equations (Neural ODEs), have received considerable attention over the last several years.
Despite their burgeoning impact, there is a lack of formal analysis techniques for these systems.
We introduce a novel reachability framework that allows for the formal analysis of their behavior.
arXiv Detail & Related papers (2022-07-13T22:05:52Z) - Neuromorphic Artificial Intelligence Systems [58.1806704582023]
Modern AI systems, based on von Neumann architecture and classical neural networks, have a number of fundamental limitations in comparison with the brain.
This article discusses such limitations and the ways they can be mitigated.
It presents an overview of currently available neuromorphic AI projects in which these limitations are overcome.
arXiv Detail & Related papers (2022-05-25T20:16:05Z) - An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols.
We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
arXiv Detail & Related papers (2021-08-09T17:39:09Z) - Bottom-up and top-down approaches for the design of neuromorphic
processing systems: Tradeoffs and synergies between natural and artificial
intelligence [3.874729481138221]
Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance.
One of these avenues is the exploration of alternative brain-inspired computing architectures that aim at achieving the flexibility and computational efficiency of biological neural processing systems.
We provide a comprehensive overview of the field, highlighting the different levels of granularity at which this paradigm shift is realized.
arXiv Detail & Related papers (2021-06-02T16:51:45Z) - Spiking Neural Networks Hardware Implementations and Challenges: a
Survey [53.429871539789445]
Spiking Neural Networks are cognitive algorithms mimicking neuron and synapse operational principles.
We present the state of the art of hardware implementations of spiking neural networks.
We discuss the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level.
arXiv Detail & Related papers (2020-05-04T13:24:00Z) - Neuromorphic Nearest-Neighbor Search Using Intel's Pohoiki Springs [3.571324984762197]
In the brain, billions of interconnected neurons perform rapid computations at extremely low energy levels.
Here, we showcase the Pohoiki Springs neuromorphic system, a mesh of 768 interconnected Loihi chips that collectively implement 100 million spiking neurons in silicon.
We demonstrate a scalable approximate k-nearest neighbor (k-NN) algorithm for searching large databases that exploits neuromorphic principles.
arXiv Detail & Related papers (2020-04-27T10:23:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.