Consistent Joint Decision-Making with Heterogeneous Learning Models
- URL: http://arxiv.org/abs/2402.03728v1
- Date: Tue, 6 Feb 2024 05:50:04 GMT
- Title: Consistent Joint Decision-Making with Heterogeneous Learning Models
- Authors: Hossein Rajaby Faghihi and Parisa Kordjamshidi
- Abstract summary: This paper introduces a novel decision-making framework that promotes consistency among decisions made by diverse models.
We map predictions from various models into globally normalized and comparable values by incorporating information about decisions' prior probability, confidence (uncertainty) and the models' expected accuracy.
- Score: 26.369155875802807
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper introduces a novel decision-making framework that promotes
consistency among decisions made by diverse models while utilizing external
knowledge. Leveraging the Integer Linear Programming (ILP) framework, we map
predictions from various models into globally normalized and comparable values
by incorporating information about decisions' prior probability, confidence
(uncertainty), and the models' expected accuracy. Our empirical study
demonstrates the superiority of our approach over conventional baselines on
multiple datasets.
Related papers
- Generative Intrinsic Optimization: Intrinsic Control with Model Learning [5.439020425819001]
Future sequence represents the outcome after executing the action into the environment.
Explicit outcomes may vary across state, return, or trajectory serving different purposes such as credit assignment or imitation learning.
We propose a policy scheme that seamlessly incorporates the mutual information, ensuring convergence to the optimal policy.
arXiv Detail & Related papers (2023-10-12T07:50:37Z) - Risk Aware Benchmarking of Large Language Models [36.95053112313244]
We propose a distributional framework for benchmarking socio-technical risks of foundation models with quantified statistical significance.
We show that the second order statistics in this test are linked to mean-risk models commonly used in econometrics and mathematical finance.
We use our framework to compare various large language models regarding risks related to drifting from instructions and outputting toxic content.
arXiv Detail & Related papers (2023-10-11T02:08:37Z) - Beyond Normal: On the Evaluation of Mutual Information Estimators [52.85079110699378]
We show how to construct a diverse family of distributions with known ground-truth mutual information.
We provide guidelines for practitioners on how to select appropriate estimator adapted to the difficulty of problem considered.
arXiv Detail & Related papers (2023-06-19T17:26:34Z) - Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation [0.0]
Cross-validation techniques for risk estimation and model selection are widely used in statistics and machine learning.
This paper presents learning via model selection with cross-validation risk estimation as a general systematic learning framework.
arXiv Detail & Related papers (2023-03-15T17:18:31Z) - Federated Learning Aggregation: New Robust Algorithms with Guarantees [63.96013144017572]
Federated learning has been recently proposed for distributed model training at the edge.
This paper presents a complete general mathematical convergence analysis to evaluate aggregation strategies in a federated learning framework.
We derive novel aggregation algorithms which are able to modify their model architecture by differentiating client contributions according to the value of their losses.
arXiv Detail & Related papers (2022-05-22T16:37:53Z) - Model-Based Deep Learning: On the Intersection of Deep Learning and
Optimization [101.32332941117271]
Decision making algorithms are used in a multitude of different applications.
Deep learning approaches that use highly parametric architectures tuned from data without relying on mathematical models are becoming increasingly popular.
Model-based optimization and data-centric deep learning are often considered to be distinct disciplines.
arXiv Detail & Related papers (2022-05-05T13:40:08Z) - Characterizing Fairness Over the Set of Good Models Under Selective
Labels [69.64662540443162]
We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance.
We provide tractable algorithms to compute the range of attainable group-level predictive disparities.
We extend our framework to address the empirically relevant challenge of selectively labelled data.
arXiv Detail & Related papers (2021-01-02T02:11:37Z) - Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets.
Part of the challenge of learning robust models lies in the influence of unobserved confounders.
We demonstrate the empirical performance of our approach on healthcare data from different modalities.
arXiv Detail & Related papers (2020-07-21T08:18:06Z) - Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
We show that a posterior approximation distinct from the variational distribution should be used for making decisions.
Motivated by these theoretical results, we propose learning several approximate proposals for the best model.
In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
arXiv Detail & Related papers (2020-02-17T19:23:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.