Solving Boltzmann Optimization Problems with Deep Learning

• URL: http://arxiv.org/abs/2401.17408v1
• Date: Tue, 30 Jan 2024 19:52:02 GMT
• Title: Solving Boltzmann Optimization Problems with Deep Learning
• Authors: Fiona Knoll, John T. Daly, Jess J. Meyer
• Abstract summary: The Ising model shows particular promise as a future framework for highly energy efficient computation. Ising systems are able to operate at energies approaching thermodynamic limits for energy consumption of computation. The challenge in creating Ising-based hardware is in optimizing useful circuits that produce correct results on fundamentally nondeterministic hardware.
• Score: 0.21485350418225244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decades of exponential scaling in high performance computing (HPC) efficiency is coming to an end. Transistor based logic in complementary metal-oxide semiconductor (CMOS) technology is approaching physical limits beyond which further miniaturization will be impossible. Future HPC efficiency gains will necessarily rely on new technologies and paradigms of compute. The Ising model shows particular promise as a future framework for highly energy efficient computation. Ising systems are able to operate at energies approaching thermodynamic limits for energy consumption of computation. Ising systems can function as both logic and memory. Thus, they have the potential to significantly reduce energy costs inherent to CMOS computing by eliminating costly data movement. The challenge in creating Ising-based hardware is in optimizing useful circuits that produce correct results on fundamentally nondeterministic hardware. The contribution of this paper is a novel machine learning approach, a combination of deep neural networks and random forests, for efficiently solving optimization problems that minimize sources of error in the Ising model. In addition, we provide a process to express a Boltzmann probability optimization problem as a supervised machine learning problem.

Related papers

• DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs [70.91804882618243]
  This paper proposes DSMoE, a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks. We implement adaptive expert routing using sigmoid activation and straight-through estimators, enabling tokens to flexibly access different aspects of model knowledge. Experiments on LLaMA models demonstrate that under equivalent computational constraints, DSMoE achieves superior performance compared to existing pruning and MoE approaches.
  arXiv  Detail & Related papers  (2025-02-18T02:37:26Z)
• OscNet: Machine Learning on CMOS Oscillator Networks [0.0]
  We propose a new and energy efficient machine learning framework implemented on CMOS Networks (OscNet) We model the developmental processes of the prenatal brain's visual system using OscNet, updating based on the biologically inspired Hebbian rule. Experimental results demonstrate that Hebbian learning pipeline on OscNet achieves performance comparable to or even surpassing traditional machine learning algorithms.
  arXiv  Detail & Related papers  (2025-02-11T02:32:32Z)
• Gradual Optimization Learning for Conformational Energy Minimization [69.36925478047682]
  Gradual Optimization Learning Framework (GOLF) for energy minimization with neural networks significantly reduces the required additional data. Our results demonstrate that the neural network trained with GOLF performs on par with the oracle on a benchmark of diverse drug-like molecules.
  arXiv  Detail & Related papers  (2023-11-05T11:48:08Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Efficient Neural PDE-Solvers using Quantization Aware Training [71.0934372968972]
  We show that quantization can successfully lower the computational cost of inference while maintaining performance. Our results on four standard PDE datasets and three network architectures show that quantization-aware training works across settings and three orders of FLOPs magnitudes.
  arXiv  Detail & Related papers  (2023-08-14T09:21:19Z)
• Energy-frugal and Interpretable AI Hardware Design using Learning Automata [5.514795777097036]
  A new machine learning algorithm, called the Tsetlin machine, has been proposed. In this paper, we investigate methods of energy-frugal artificial intelligence hardware design. We show that frugal resource allocation can provide decisive energy reduction while also achieving robust and interpretable learning.
  arXiv  Detail & Related papers  (2023-05-19T15:11:18Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• A full-stack view of probabilistic computing with p-bits: devices, architectures and algorithms [0.014319921806060482]
  We provide a full-stack review of probabilistic computing with p-bits. We argue that p-bits could be used to build energy-efficient probabilistic systems. We outline the main applications of probabilistic computers ranging from machine learning to AI.
  arXiv  Detail & Related papers  (2023-02-13T15:36:07Z)
• Unsupervised Optimal Power Flow Using Graph Neural Networks [172.33624307594158]
  We use a graph neural network to learn a nonlinear parametrization between the power demanded and the corresponding allocation. We show through simulations that the use of GNNs in this unsupervised learning context leads to solutions comparable to standard solvers.
  arXiv  Detail & Related papers  (2022-10-17T17:30:09Z)
• Neuromorphic scaling advantages for energy-efficient random walk computation [0.28144129864580447]
  Neuromorphic computing aims to replicate the brain's computational structure and architecture in man-made hardware. We show that high-degree parallelism and configurability of spiking neuromorphic architectures makes them well-suited to implement random walks via discrete time chains. We find that NMC platforms, at a sufficient scale, can drastically reduce the energy demands of high-performance computing platforms.
  arXiv  Detail & Related papers  (2021-07-27T19:44:33Z)
• From DNNs to GANs: Review of efficient hardware architectures for deep learning [0.0]
  Neural network and deep learning has been started to impact the present research paradigm. DSP processors are incapable of performing neural network, activation function, convolutional neural network and generative adversarial network operations. Different algorithms have been adapted to design a DSP processor compatible for fast performance in neural network, activation function, convolutional neural network and generative adversarial network.
  arXiv  Detail & Related papers  (2021-06-06T13:23:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.