E2E-AT: A Unified Framework for Tackling Uncertainty in Task-aware End-to-end Learning

• URL: http://arxiv.org/abs/2312.10587v2
• Date: Sat, 23 Dec 2023 16:38:26 GMT
• Title: E2E-AT: A Unified Framework for Tackling Uncertainty in Task-aware End-to-end Learning
• Authors: Wangkun Xu and Jianhong Wang and Fei Teng
• Abstract summary: We propose a unified framework that covers the uncertainties emerging in both the input feature space of the machine learning models and the constrained optimization models. We show that neglecting the uncertainty of COs during training causes a new trigger for generalization errors. The framework is described as a robust optimization problem and is practically solved via end-to-end adversarial training (E2E-AT)
• Score: 9.741277008050927
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Successful machine learning involves a complete pipeline of data, model, and downstream applications. Instead of treating them separately, there has been a prominent increase of attention within the constrained optimization (CO) and machine learning (ML) communities towards combining prediction and optimization models. The so-called end-to-end (E2E) learning captures the task-based objective for which they will be used for decision making. Although a large variety of E2E algorithms have been presented, it has not been fully investigated how to systematically address uncertainties involved in such models. Most of the existing work considers the uncertainties of ML in the input space and improves robustness through adversarial training. We extend this idea to E2E learning and prove that there is a robustness certification procedure by solving augmented integer programming. Furthermore, we show that neglecting the uncertainty of COs during training causes a new trigger for generalization errors. To include all these components, we propose a unified framework that covers the uncertainties emerging in both the input feature space of the ML models and the COs. The framework is described as a robust optimization problem and is practically solved via end-to-end adversarial training (E2E-AT). Finally, the performance of E2E-AT is evaluated by a real-world end-to-end power system operation problem, including load forecasting and sequential scheduling tasks.

Related papers

• SMLE: Safe Machine Learning via Embedded Overapproximation [4.129133569151574]
  We consider the task of training differentiable ML models guaranteed to satisfy designer-chosen properties. This is very challenging, due to the computational complexity of rigorously verifying and enforcing compliance in modern neural models. We provide an innovative approach based on three components: 1) a general, simple architecture enabling efficient verification with a conservative semantic. We evaluate our approach on properties defined by linear inequalities in regression, and on mutually exclusive classes in multilabel classification.
  arXiv  Detail & Related papers  (2024-09-30T17:19:57Z)
• Exploring End-to-end Differentiable Neural Charged Particle Tracking -- A Loss Landscape Perspective [0.0]
  We propose an E2E differentiable decision-focused learning scheme for particle tracking. We show that differentiable variations of discrete assignment operations allows for efficient network optimization. We argue that E2E differentiability provides, besides the general availability of gradient information, an important tool for robust particle tracking to mitigate prediction instabilities.
  arXiv  Detail & Related papers  (2024-07-18T11:42:58Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making. We present a process-based benchmark MR-Ben that demands a meta-reasoning skill. Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• Rethinking Resource Management in Edge Learning: A Joint Pre-training and Fine-tuning Design Paradigm [87.47506806135746]
  In some applications, edge learning is experiencing a shift in focusing from conventional learning from scratch to new two-stage learning. This paper considers the problem of joint communication and computation resource management in a two-stage edge learning system. It is shown that the proposed joint resource management over the pre-training and fine-tuning stages well balances the system performance trade-off.
  arXiv  Detail & Related papers  (2024-04-01T00:21:11Z)
• Near-Optimal Solutions of Constrained Learning Problems [85.48853063302764]
  In machine learning systems, the need to curtail their behavior has become increasingly apparent. This is evidenced by recent advancements towards developing models that satisfy dual robustness variables. Our results show that rich parametrizations effectively mitigate non-dimensional, finite learning problems.
  arXiv  Detail & Related papers  (2024-03-18T14:55:45Z)
• Learning Constrained Optimization with Deep Augmented Lagrangian Methods [54.22290715244502]
  A machine learning (ML) model is trained to emulate a constrained optimization solver. This paper proposes an alternative approach, in which the ML model is trained to predict dual solution estimates directly. It enables an end-to-end training scheme is which the dual objective is as a loss function, and solution estimates toward primal feasibility, emulating a Dual Ascent method.
  arXiv  Detail & Related papers  (2024-03-06T04:43:22Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Efficient Knowledge Compilation Beyond Weighted Model Counting [7.828647825246474]
  We introduce Second Level Algebraic Model Counting (2AMC) as a generic framework for these kinds of problems. First level techniques based on Knowledge Compilation (KC) have been adapted for specific 2AMC instances by imposing variable order constraints. We show that we can exploit the logical structure of a 2AMC problem to omit parts of these constraints, thus limiting the negative effect.
  arXiv  Detail & Related papers  (2022-05-16T08:10:40Z)
• The Perils of Learning Before Optimizing [16.97597806975415]
  We show how prediction models can be learned end-to-end by differentiating through the optimization task. We show that the performance gap between a two-stage and end-to-end approach is closely related to the emphprice of correlation concept in optimization.
  arXiv  Detail & Related papers  (2021-06-18T20:43:47Z)
• Combining Deep Learning and Optimization for Security-Constrained Optimal Power Flow [94.24763814458686]
  Security-constrained optimal power flow (SCOPF) is fundamental in power systems. Modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs. This paper proposes a novel approach that combines deep learning and robust optimization techniques.
  arXiv  Detail & Related papers  (2020-07-14T12:38:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.