Evaluating Bayesian Model Visualisations
- URL: http://arxiv.org/abs/2201.03604v1
- Date: Mon, 10 Jan 2022 19:15:39 GMT
- Title: Evaluating Bayesian Model Visualisations
- Authors: Sebastian Stein (1), John H. Williamson (1) ((1) School of Computing
Science, University of Glasgow, Scotland, United Kingdom)
- Abstract summary: Probabilistic models inform an increasingly broad range of business and policy decisions ultimately made by people.
Recent algorithmic, computational, and software framework development progress facilitate the proliferation of Bayesian probabilistic models.
While they can empower decision makers to explore complex queries and to perform what-if-style conditioning in theory, suitable visualisations and interactive tools are needed to maximise users' comprehension and rational decision making under uncertainty.
- Score: 0.39845810840390733
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Probabilistic models inform an increasingly broad range of business and
policy decisions ultimately made by people. Recent algorithmic, computational,
and software framework development progress facilitate the proliferation of
Bayesian probabilistic models, which characterise unobserved parameters by
their joint distribution instead of point estimates. While they can empower
decision makers to explore complex queries and to perform what-if-style
conditioning in theory, suitable visualisations and interactive tools are
needed to maximise users' comprehension and rational decision making under
uncertainty. In this paper, propose a protocol for quantitative evaluation of
Bayesian model visualisations and introduce a software framework implementing
this protocol to support standardisation in evaluation practice and facilitate
reproducibility. We illustrate the evaluation and analysis workflow on a user
study that explores whether making Boxplots and Hypothetical Outcome Plots
interactive can increase comprehension or rationality and conclude with design
guidelines for researchers looking to conduct similar studies in the future.
Related papers
- A Bayesian Approach to Robust Inverse Reinforcement Learning [54.24816623644148]
We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL)
The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics.
Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed to have a highly accurate model of the environment.
arXiv Detail & Related papers (2023-09-15T17:37:09Z) - Communicating Uncertainty in Machine Learning Explanations: A
Visualization Analytics Approach for Predictive Process Monitoring [0.0]
This study explores how model uncertainty can be effectively communicated in global and local post-hoc explanation approaches.
By combining these two research directions, decision-makers can not only justify the plausibility of explanation-driven actionable insights but also validate their reliability.
arXiv Detail & Related papers (2023-04-12T09:44:32Z) - A review of predictive uncertainty estimation with machine learning [0.0]
We review the topic of predictive uncertainty estimation with machine learning algorithms.
We discuss the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions.
The review expedites our understanding on how to develop new algorithms tailored to users' needs.
arXiv Detail & Related papers (2022-09-17T10:36:30Z) - Bayesian Graph Contrastive Learning [55.36652660268726]
We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders.
Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector.
We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
arXiv Detail & Related papers (2021-12-15T01:45:32Z) - Deceptive Decision-Making Under Uncertainty [25.197098169762356]
We study the design of autonomous agents that are capable of deceiving outside observers about their intentions while carrying out tasks.
By modeling the agent's behavior as a Markov decision process, we consider a setting where the agent aims to reach one of multiple potential goals.
We propose a novel approach to model observer predictions based on the principle of maximum entropy and to efficiently generate deceptive strategies.
arXiv Detail & Related papers (2021-09-14T14:56:23Z) - Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
We propose a deep co-attention network for multi-view subspace learning.
It aims to extract both the common information and the complementary information in an adversarial setting.
In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
arXiv Detail & Related papers (2021-02-15T18:46:44Z) - (Un)fairness in Post-operative Complication Prediction Models [20.16366948502659]
We consider a real-life example of risk estimation before surgery and investigate the potential for bias or unfairness of a variety of algorithms.
Our approach creates transparent documentation of potential bias so that the users can apply the model carefully.
arXiv Detail & Related papers (2020-11-03T22:11:19Z) - Forethought and Hindsight in Credit Assignment [62.05690959741223]
We work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models.
We investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated.
arXiv Detail & Related papers (2020-10-26T16:00:47Z) - Evaluating Interactive Summarization: an Expansion-Based Framework [97.0077722128397]
We develop an end-to-end evaluation framework for interactive summarization.
Our framework includes a procedure of collecting real user sessions and evaluation measures relying on standards.
All of our solutions are intended to be released publicly as a benchmark.
arXiv Detail & Related papers (2020-09-17T15:48:13Z) - Plausible Counterfactuals: Auditing Deep Learning Classifiers with
Realistic Adversarial Examples [84.8370546614042]
Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data.
Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model.
Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
arXiv Detail & Related papers (2020-03-25T11:08:56Z) - Asking the Right Questions: Learning Interpretable Action Models Through
Query Answering [33.08099403894141]
This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act.
Our main contributions are a new paradigm for estimating such models using a minimal query interface with the agent, and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model.
arXiv Detail & Related papers (2019-12-29T09:05:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.