Learning to Guide Human Experts via Personalized Large Language Models

• URL: http://arxiv.org/abs/2308.06039v1
• Date: Fri, 11 Aug 2023 09:36:33 GMT
• Title: Learning to Guide Human Experts via Personalized Large Language Models
• Authors: Debodeep Banerjee, Stefano Teso, Andrea Passerini
• Abstract summary: In learning to defer, a predictor identifies risky decisions and defers them to a human expert. In learning to guide, the machine provides guidance useful to guide decision-making, and the human is entirely responsible for coming up with a decision.
• Score: 23.7625973884849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In learning to defer, a predictor identifies risky decisions and defers them to a human expert. One key issue with this setup is that the expert may end up over-relying on the machine's decisions, due to anchoring bias. At the same time, whenever the machine chooses the deferral option the expert has to take decisions entirely unassisted. As a remedy, we propose learning to guide (LTG), an alternative framework in which -- rather than suggesting ready-made decisions -- the machine provides guidance useful to guide decision-making, and the human is entirely responsible for coming up with a decision. We also introduce SLOG, an LTG implementation that leverages (a small amount of) human supervision to convert a generic large language model into a module capable of generating textual guidance, and present preliminary but promising results on a medical diagnosis task.

Related papers

• Learning To Guide Human Decision Makers With Vision-Language Models [17.957952996809716]
  There is increasing interest in developing AIs for assisting human decision-making in high-stakes tasks, such as medical diagnosis. We introduce learning to guide (LTG), an alternative framework in which - rather than taking control from the human expert - the machine provides guidance. In order to ensure guidance is interpretable, we develop SLOG, an approach for turning any vision-language model into a capable generator of textual guidance.
  arXiv  Detail & Related papers  (2024-03-25T07:34:42Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• Learning Personalized Decision Support Policies [56.949897454209186]
  $texttModiste$ is an interactive tool to learn personalized decision support policies. We find that personalized policies outperform offline policies, and, in the cost-aware setting, reduce the incurred cost with minimal degradation to performance.
  arXiv  Detail & Related papers  (2023-04-13T17:53:34Z)
• Predicting and Understanding Human Action Decisions during Skillful Joint-Action via Machine Learning and Explainable-AI [1.3381749415517021]
  This study uses supervised machine learning and explainable artificial intelligence to model, predict and understand human decision-making. Long short-term memory networks were trained to predict the target selection decisions of expert and novice actors completing a dyadic herding task.
  arXiv  Detail & Related papers  (2022-06-06T16:54:43Z)
• Boosting human decision-making with AI-generated decision aids [8.373151777137792]
  We developed an algorithm for translating the output of our previous method into procedural instructions. Experiments showed that these automatically generated decision-aids significantly improved people's performance in planning a road trip and choosing a mortgage. These findings suggest that AI-powered boosting might have potential for improving human decision-making in the real world.
  arXiv  Detail & Related papers  (2022-03-05T15:57:20Z)
• Decision Rule Elicitation for Domain Adaptation [93.02675868486932]
  Human-in-the-loop machine learning is widely used in artificial intelligence (AI) to elicit labels from experts. In this work, we allow experts to additionally produce decision rules describing their decision-making. We show that decision rule elicitation improves domain adaptation of the algorithm and helps to propagate expert's knowledge to the AI model.
  arXiv  Detail & Related papers  (2021-02-23T08:07:22Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• Indecision Modeling [50.00689136829134]
  It is important that AI systems act in ways which align with human values. People are often indecisive, and especially so when their decision has moral implications.
  arXiv  Detail & Related papers  (2020-12-15T18:32:37Z)
• A Bandit Model for Human-Machine Decision Making with Private Information and Opacity [16.665883787432858]
  We show a two-player learning problem where one player is the machine and the other the human. A lower bound quantifies the worst-case hardness of optimally advising a decision maker who is opaque. An upper bound shows that a simple coordination strategy is nearly minimax optimal.
  arXiv  Detail & Related papers  (2020-07-09T13:43:08Z)
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores [85.12096045419686]
  We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
  arXiv  Detail & Related papers  (2020-02-19T07:27:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.