Indecision Modeling
- URL: http://arxiv.org/abs/2012.08485v2
- Date: Fri, 12 Mar 2021 22:44:37 GMT
- Title: Indecision Modeling
- Authors: Duncan C McElfresh, Lok Chan, Kenzie Doyle, Walter Sinnott-Armstrong,
Vincent Conitzer, Jana Schaich Borg, John P Dickerson
- Abstract summary: It is important that AI systems act in ways which align with human values.
People are often indecisive, and especially so when their decision has moral implications.
- Score: 50.00689136829134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: AI systems are often used to make or contribute to important decisions in a
growing range of applications, including criminal justice, hiring, and
medicine. Since these decisions impact human lives, it is important that the AI
systems act in ways which align with human values. Techniques for preference
modeling and social choice help researchers learn and aggregate peoples'
preferences, which are used to guide AI behavior; thus, it is imperative that
these learned preferences are accurate. These techniques often assume that
people are willing to express strict preferences over alternatives; which is
not true in practice. People are often indecisive, and especially so when their
decision has moral implications. The philosophy and psychology literature shows
that indecision is a measurable and nuanced behavior -- and that there are
several different reasons people are indecisive. This complicates the task of
both learning and aggregating preferences, since most of the relevant
literature makes restrictive assumptions on the meaning of indecision. We begin
to close this gap by formalizing several mathematical \emph{indecision} models
based on theories from philosophy, psychology, and economics; these models can
be used to describe (indecisive) agent decisions, both when they are allowed to
express indecision and when they are not. We test these models using data
collected from an online survey where participants choose how to
(hypothetically) allocate organs to patients waiting for a transplant.
Related papers
- Exploring and steering the moral compass of Large Language Models [55.2480439325792]
Large Language Models (LLMs) have become central to advancing automation and decision-making across various sectors.
This study proposes a comprehensive comparative analysis of the most advanced LLMs to assess their moral profiles.
arXiv Detail & Related papers (2024-05-27T16:49:22Z) - Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies [0.43981305860983716]
We show how to compare the performance of three alternative decision-making systems--human-alone, human-with-AI, and AI-alone.
We find that the risk assessment recommendations do not improve the classification accuracy of a judge's decision to impose cash bail.
arXiv Detail & Related papers (2024-03-18T01:04:52Z) - Decision Theoretic Foundations for Experiments Evaluating Human Decisions [18.27590643693167]
We argue that to attribute loss in human performance to forms of bias, an experiment must provide participants with the information that a rational agent would need to identify the utility-maximizing decision.
As a demonstration, we evaluate the extent to which recent evaluations of decision-making from the literature on AI-assisted decisions achieve these criteria.
arXiv Detail & Related papers (2024-01-25T16:21:37Z) - From DDMs to DNNs: Using process data and models of decision-making to
improve human-AI interactions [1.1510009152620668]
We argue that artificial intelligence (AI) research would benefit from a stronger focus on insights about how decisions emerge over time.
First, we introduce a highly established computational framework that assumes decisions to emerge from the noisy accumulation of evidence.
Next, we discuss to what extent current approaches in multi-agent AI do or do not incorporate process data and models of decision making.
arXiv Detail & Related papers (2023-08-29T11:27:22Z) - In Search of Insights, Not Magic Bullets: Towards Demystification of the
Model Selection Dilemma in Heterogeneous Treatment Effect Estimation [92.51773744318119]
This paper empirically investigates the strengths and weaknesses of different model selection criteria.
We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them.
arXiv Detail & Related papers (2023-02-06T16:55:37Z) - Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have
to Act Randomly and Society Seems to Accept This [0.8889304968879161]
We feel that akin to human decisions, judgments of artificial agents should necessarily be grounded in some moral principles.
Yet a decision-maker can only make truly ethical (based on any ethical theory) and fair (according to any notion of fairness) decisions if full information on all the relevant factors on which the decision is based are available at the time of decision-making.
arXiv Detail & Related papers (2021-11-15T05:39:02Z) - Learning the Preferences of Uncertain Humans with Inverse Decision
Theory [10.926992035470372]
We study the setting of inverse decision theory (IDT), a framework where a human is observed making non-sequential binary decisions under uncertainty.
In IDT, the human's preferences are conveyed through their loss function, which expresses a tradeoff between different types of mistakes.
We show that it is actually easier to identify preferences when the decision problem is more uncertain.
arXiv Detail & Related papers (2021-06-19T00:11:13Z) - Individual Explanations in Machine Learning Models: A Survey for
Practitioners [69.02688684221265]
The use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise.
Many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways.
Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models.
arXiv Detail & Related papers (2021-04-09T01:46:34Z) - Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap.
We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert.
Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
arXiv Detail & Related papers (2021-01-24T05:40:29Z) - Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life
Anecdotes [72.64975113835018]
Motivated by descriptive ethics, we investigate a novel, data-driven approach to machine ethics.
We introduce Scruples, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes.
Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement.
arXiv Detail & Related papers (2020-08-20T17:34:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.