Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach

• URL: http://arxiv.org/abs/2410.03085v1
• Date: Fri, 4 Oct 2024 02:10:20 GMT
• Title: Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach
• Authors: Parikshit Pareek, Kaarthik Sundar, Deepjyoti Deka, Sidhant Misra,
• Abstract summary: Constrained optimization problems arise in various engineering system operations such as inventory management electric power grids. This work introduces a learning scheme using Bayesian Networks (BNNs) to solve constrained optimization problems under limited data and restricted model times. We show that the proposed learning method outperforms conventional BNN and deep neural network (DNN) architectures.
• Score: 2.943640991628177
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Constrained optimization problems arise in various engineering system operations such as inventory management and electric power grids. However, the requirement to repeatedly solve such optimization problems with uncertain parameters poses a significant computational challenge. This work introduces a learning scheme using Bayesian Neural Networks (BNNs) to solve constrained optimization problems under limited labeled data and restricted model training times. We propose a semi-supervised BNN for this practical but complex regime, wherein training commences in a sandwiched fashion, alternating between a supervised learning step (using labeled data) for minimizing cost, and an unsupervised learning step (using unlabeled data) for enforcing constraint feasibility. Both supervised and unsupervised steps use a Bayesian approach, where Stochastic Variational Inference is employed for approximate Bayesian inference. We show that the proposed semi-supervised learning method outperforms conventional BNN and deep neural network (DNN) architectures on important non-convex constrained optimization problems from energy network operations, achieving up to a tenfold reduction in expected maximum equality gap and halving the optimality and inequality (feasibility) gaps, without requiring any correction or projection step. By leveraging the BNN's ability to provide posterior samples at minimal computational cost, we demonstrate that a Selection via Posterior (SvP) scheme can further reduce equality gaps by more than 10%. We also provide tight and practically meaningful probabilistic confidence bounds that can be constructed using a low number of labeled testing data and readily adapted to other applications.

Related papers

• QP-SNN: Quantized and Pruned Spiking Neural Networks [10.74122828236122]
  Spiking Neural Networks (SNNs) leverage spikes to encode information and operate in an event-driven manner. We propose a hardware-friendly and lightweight SNN, aimed at effectively deploying high-performance SNN in resource-limited scenarios.
  arXiv  Detail & Related papers  (2025-02-09T13:50:59Z)
• Reliable Projection Based Unsupervised Learning for Semi-Definite QCQP with Application of Beamforming Optimization [11.385703484113552]
  In this paper, we investigate a special class of quadratic (QCQP) with semi-definite constraints. We propose a neural network (NN) as a promising method to obtain a high-performing constraint solution. Unsupervised learning is used, so the NN can be effectively efficiently without labels.
  arXiv  Detail & Related papers  (2024-07-04T06:26:01Z)
• Bayesian Entropy Neural Networks for Physics-Aware Prediction [14.705526856205454]
  We introduce BENN, a framework designed to impose constraints on Bayesian Neural Network (BNN) predictions. Benn is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. Results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.
  arXiv  Detail & Related papers  (2024-07-01T07:00:44Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Scalable Resource Management for Dynamic MEC: An Unsupervised Link-Output Graph Neural Network Approach [36.32772317151467]
  Deep learning has been successfully adopted in mobile edge computing (MEC) to optimize task offloading and resource allocation. The dynamics of edge networks raise two challenges in neural network (NN)-based optimization methods: low scalability and high training costs. In this paper, a novel link-output GNN (LOGNN)-based resource management approach is proposed to flexibly optimize the resource allocation in MEC.
  arXiv  Detail & Related papers  (2023-06-15T08:21:41Z)
• Enriching Neural Network Training Dataset to Improve Worst-Case Performance Guarantees [0.0]
  We show that adapting the NN training dataset during training can improve the NN performance and substantially reduce its worst-case violations. This paper proposes an algorithm that identifies and enriches the training dataset with critical datapoints that reduce the worst-case violations and deliver a neural network with improved worst-case performance guarantees.
  arXiv  Detail & Related papers  (2023-03-23T12:59:37Z)
• Unsupervised Optimal Power Flow Using Graph Neural Networks [172.33624307594158]
  We use a graph neural network to learn a nonlinear parametrization between the power demanded and the corresponding allocation. We show through simulations that the use of GNNs in this unsupervised learning context leads to solutions comparable to standard solvers.
  arXiv  Detail & Related papers  (2022-10-17T17:30:09Z)
• Recurrent Bilinear Optimization for Binary Neural Networks [58.972212365275595]
  BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors. Our work is the first attempt to optimize BNNs from the bilinear perspective. We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets.
  arXiv  Detail & Related papers  (2022-09-04T06:45:33Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Learning to Solve the AC-OPF using Sensitivity-Informed Deep Neural Networks [52.32646357164739]
  We propose a deep neural network (DNN) to solve the solutions of the optimal power flow (ACOPF) The proposed SIDNN is compatible with a broad range of OPF schemes. It can be seamlessly integrated in other learning-to-OPF schemes.
  arXiv  Detail & Related papers  (2021-03-27T00:45:23Z)
• S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration [74.5509794733707]
  We present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution. Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.515% on BNNs. Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods.
  arXiv  Detail & Related papers  (2021-02-17T18:59:28Z)
• Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation [101.22379613810881]
  We consider data-driven optimization problems where one must maximize a function given only queries at a fixed set of points. This problem setting emerges in many domains where function evaluation is a complex and expensive process. We propose a tractable approximation that allows us to scale our method to high-capacity neural network models.
  arXiv  Detail & Related papers  (2021-02-16T06:04:27Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Combining Deep Learning and Optimization for Security-Constrained Optimal Power Flow [94.24763814458686]
  Security-constrained optimal power flow (SCOPF) is fundamental in power systems. Modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs. This paper proposes a novel approach that combines deep learning and robust optimization techniques.
  arXiv  Detail & Related papers  (2020-07-14T12:38:21Z)
• Unsupervised Deep Learning for Optimizing Wireless Systems with Instantaneous and Statistic Constraints [29.823814915538463]
  We establish a unified framework of using unsupervised deep learning to solve both kinds of problems with both instantaneous and statistic constraints. We show that unsupervised learning outperforms supervised learning in terms of violation probability and approximation accuracy of the optimal policy.
  arXiv  Detail & Related papers  (2020-05-30T13:37:14Z)
• Nonconvex sparse regularization for deep neural networks and its optimality [1.9798034349981162]
  Deep neural network (DNN) estimators can attain optimal convergence rates for regression and classification problems. We propose a novel penalized estimation method for sparse DNNs. We prove that the sparse-penalized estimator can adaptively attain minimax convergence rates for various nonparametric regression problems.
  arXiv  Detail & Related papers  (2020-03-26T07:15:28Z)
• Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning [96.01176486957226]
  Resource allocation and transceivers in wireless networks are usually designed by solving optimization problems. In this article, we introduce unsupervised and reinforced-unsupervised learning frameworks for solving both variable and functional optimization problems.
  arXiv  Detail & Related papers  (2020-01-03T11:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.