Effects of AI Feedback on Learning, the Skill Gap, and Intellectual Diversity

• URL: http://arxiv.org/abs/2409.18660v1
• Date: Fri, 27 Sep 2024 11:44:03 GMT
• Title: Effects of AI Feedback on Learning, the Skill Gap, and Intellectual Diversity
• Authors: Christoph Riedl, Eric Bogert,
• Abstract summary: We investigate how AI use affects three interrelated long-term outcomes: learning, skill gap, and diversity of decision strategies. We show that individuals are far more likely to seek AI feedback in situations in which they experienced success rather than failure. As a result, access to AI feedback increases, rather than decreases, the skill gap between high- and low-skilled individuals.
• Score: 4.8342038441006805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Can human decision-makers learn from AI feedback? Using data on 52,000 decision-makers from a large online chess platform, we investigate how their AI use affects three interrelated long-term outcomes: Learning, skill gap, and diversity of decision strategies. First, we show that individuals are far more likely to seek AI feedback in situations in which they experienced success rather than failure. This AI feedback seeking strategy turns out to be detrimental to learning: Feedback on successes decreases future performance, while feedback on failures increases it. Second, higher-skilled decision-makers seek AI feedback more often and are far more likely to seek AI feedback after a failure, and benefit more from AI feedback than lower-skilled individuals. As a result, access to AI feedback increases, rather than decreases, the skill gap between high- and low-skilled individuals. Finally, we leverage 42 major platform updates as natural experiments to show that access to AI feedback causes a decrease in intellectual diversity of the population as individuals tend to specialize in the same areas. Together, those results indicate that learning from AI feedback is not automatic and using AI correctly seems to be a skill itself. Furthermore, despite its individual-level benefits, access to AI feedback can have significant population-level downsides including loss of intellectual diversity and an increasing skill gap.

Related papers

• Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public [46.86429592892395]
  explainable AI (XAI) addresses this by providing AI decision-making insight.<n>We present results from two large-scale experiments combining a fairness-based diagnosis AI model and different XAI explanations.
  arXiv  Detail & Related papers  (2025-12-14T00:06:06Z)
• Engaging with AI: How Interface Design Shapes Human-AI Collaboration in High-Stakes Decision-Making [8.948482790298645]
  We examine how various decision-support mechanisms impact user engagement, trust, and human-AI collaborative task performance. Our findings reveal that mechanisms like AI confidence levels, text explanations, and performance visualizations enhanced human-AI collaborative task performance.
  arXiv  Detail & Related papers  (2025-01-28T02:03:00Z)
• Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making [57.53469908423318]
  We show the effects of performance pressure on AI advice reliance when laypeople complete a common AI-assisted task. We find that when the stakes are high, people use AI advice more appropriately than when stakes are lower, regardless of the presence of an AI explanation.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills [24.04643864795939]
  People's decision-making abilities often fail to improve when they rely on AI for decision-support. Most AI systems offer "unilateral" explanations that justify the AI's decision but do not account for users' thinking. We introduce a framework for generating human-centered contrastive explanations that explain the difference between AI's choice and a predicted, likely human choice.
  arXiv  Detail & Related papers  (2024-10-05T18:21:04Z)
• Navigating AI Fallibility: Examining People's Reactions and Perceptions of AI after Encountering Personality Misrepresentations [7.256711790264119]
  Hyper-personalized AI systems profile people's characteristics to provide personalized recommendations. These systems are not immune to errors when making inferences about people's most personal traits. We present two studies to examine how people react and perceive AI after encountering personality misrepresentations.
  arXiv  Detail & Related papers  (2024-05-25T21:27:15Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Improving Human-AI Collaboration With Descriptions of AI Behavior [14.904401331154062]
  People work with AI systems to improve their decision making, but often under- or over-rely on AI predictions and perform worse than they would have unassisted. To help people appropriately rely on AI aids, we propose showing them behavior descriptions.
  arXiv  Detail & Related papers  (2023-01-06T00:33:08Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making [4.877174544937129]
  People supported by AI-powered decision support tools frequently overrely on the AI. Adding explanations to the AI decisions does not appear to reduce the overreliance. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.
  arXiv  Detail & Related papers  (2021-02-19T00:38:53Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.