Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationales

• URL: http://arxiv.org/abs/2305.07095v1
• Date: Thu, 11 May 2023 19:01:13 GMT
• Title: Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationales
• Authors: Brihi Joshi, Ziyi Liu, Sahana Ramnath, Aaron Chan, Zhewei Tong, Shaoliang Nie, Qifan Wang, Yejin Choi, Xiang Ren
• Abstract summary: We show that human utility of existing rationales is far from satisfactory, and expensive to estimate with human studies. We translate this finding into an automated score, GEN-U, that can help improve LMs' ability to generate rationales with better human utility.
• Score: 62.02328001381361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Among the remarkable emergent capabilities of large language models (LMs) is free-text rationalization; beyond a certain scale, large LMs are capable of generating seemingly useful rationalizations, which in turn, can dramatically enhance their performances on leaderboards. This phenomenon raises a question: can machine generated rationales also be useful for humans, especially when lay humans try to answer questions based on those machine rationales? We observe that human utility of existing rationales is far from satisfactory, and expensive to estimate with human studies. Existing metrics like task performance of the LM generating the rationales, or similarity between generated and gold rationales are not good indicators of their human utility. While we observe that certain properties of rationales like conciseness and novelty are correlated with their human utility, estimating them without human involvement is challenging. We show that, by estimating a rationale's helpfulness in answering similar unseen instances, we can measure its human utility to a better extent. We also translate this finding into an automated score, GEN-U, that we propose, which can help improve LMs' ability to generate rationales with better human utility, while maintaining most of its task performance. Lastly, we release all code and collected data with this project.

Related papers

• Can Language Models Learn to Skip Steps? [59.84848399905409]
  We study the ability to skip steps in reasoning. Unlike humans, who may skip steps to enhance efficiency or to reduce cognitive load, models do not possess such motivations. Our work presents the first exploration into human-like step-skipping ability.
  arXiv  Detail & Related papers  (2024-11-04T07:10:24Z)
• WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks [85.95607119635102]
  Large language models (LLMs) can mimic human-like intelligence. WorkArena++ is designed to evaluate the planning, problem-solving, logical/arithmetic reasoning, retrieval, and contextual understanding abilities of web agents.
  arXiv  Detail & Related papers  (2024-07-07T07:15:49Z)
• Tailoring Self-Rationalizers with Multi-Reward Distillation [88.95781098418993]
  Large language models (LMs) are capable of generating free-text rationales to aid question answering. In this work, we enable small-scale LMs to generate rationales that improve downstream task performance. Our method, MaRio, is a multi-reward conditioned self-rationalization algorithm.
  arXiv  Detail & Related papers  (2023-11-06T00:20:11Z)
• Modeling Bounded Rationality in Multi-Agent Simulations Using Rationally Inattentive Reinforcement Learning [85.86440477005523]
  We study more human-like RL agents which incorporate an established model of human-irrationality, the Rational Inattention (RI) model. RIRL models the cost of cognitive information processing using mutual information. We show that using RIRL yields a rich spectrum of new equilibrium behaviors that differ from those found under rational assumptions.
  arXiv  Detail & Related papers  (2022-01-18T20:54:00Z)
• Human irrationality: both bad and good for reward inference [3.706222947143855]
  This work aims to better understand the effect irrationalities can have on reward inference. We operationalize irrationality in the language of MDPs, by altering the Bellman optimality equation. We show that an irrational human, when correctly modelled, can communicate more information about the reward than a perfectly rational human can.
  arXiv  Detail & Related papers  (2021-11-12T21:44:15Z)
• Humanly Certifying Superhuman Classifiers [8.736864280782592]
  Estimating the performance of a machine learning system is a longstanding challenge in artificial intelligence research. We develop a theory for estimating the accuracy compared to the oracle, using only imperfect human annotations for reference. Our analysis provides a simple recipe for detecting and certifying superhuman performance in this setting.
  arXiv  Detail & Related papers  (2021-09-16T11:00:05Z)
• Inductive Biases for Deep Learning of Higher-Level Cognition [108.89281493851358]
  A fascinating hypothesis is that human and animal intelligence could be explained by a few principles. This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
  arXiv  Detail & Related papers  (2020-11-30T18:29:25Z)
• Evaluating and Characterizing Human Rationales [12.678505281794715]
  We find that human rationales do not necessarily perform well on automated metrics. We propose improved metrics to account for model-dependent baseline performance. Our work leads to actionable suggestions for evaluating and characterizing rationales.
  arXiv  Detail & Related papers  (2020-10-09T18:00:04Z)
• Leveraging Rationales to Improve Human Task Performance [15.785125079811902]
  Given a computational system's performance exceeds that of its human user, can explainable AI capabilities be leveraged to improve the performance of the human? We introduce the Rationale-Generating Algorithm, an automated technique for generating rationales for utility-based computational methods. Results show that our approach produces rationales that lead to statistically significant improvement in human task performance.
  arXiv  Detail & Related papers  (2020-02-11T04:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.