Learning to Prompt in the Classroom to Understand AI Limits: A pilot
study
- URL: http://arxiv.org/abs/2307.01540v2
- Date: Fri, 1 Sep 2023 15:31:21 GMT
- Title: Learning to Prompt in the Classroom to Understand AI Limits: A pilot
study
- Authors: Emily Theophilou, Cansu Koyuturk, Mona Yavari, Sathya Bursic, Gregor
Donabauer, Alessia Telari, Alessia Testa, Raffaele Boiano, Davinia
Hernandez-Leo, Martin Ruskov, Davide Taibi, Alessandro Gabbiadini, Dimitri
Ognibene
- Abstract summary: Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems.
However, excitement has led to negative sentiments, even as AI methods demonstrate remarkable contributions.
A pilot educational intervention was performed in a high school with 21 students.
- Score: 35.06607166918901
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Artificial intelligence's (AI) progress holds great promise in tackling
pressing societal concerns such as health and climate. Large Language Models
(LLM) and the derived chatbots, like ChatGPT, have highly improved the natural
language processing capabilities of AI systems allowing them to process an
unprecedented amount of unstructured data. However, the ensuing excitement has
led to negative sentiments, even as AI methods demonstrate remarkable
contributions (e.g. in health and genetics). A key factor contributing to this
sentiment is the misleading perception that LLMs can effortlessly provide
solutions across domains, ignoring their limitations such as hallucinations and
reasoning constraints. Acknowledging AI fallibility is crucial to address the
impact of dogmatic overconfidence in possibly erroneous suggestions generated
by LLMs. At the same time, it can reduce fear and other negative attitudes
toward AI. This necessitates comprehensive AI literacy interventions that
educate the public about LLM constraints and effective usage techniques, i.e
prompting strategies. With this aim, a pilot educational intervention was
performed in a high school with 21 students. It involved presenting high-level
concepts about intelligence, AI, and LLMs, followed by practical exercises
involving ChatGPT in creating natural educational conversations and applying
established prompting strategies. Encouraging preliminary results emerged,
including high appreciation of the activity, improved interaction quality with
the LLM, reduced negative AI sentiments, and a better grasp of limitations,
specifically unreliability, limited understanding of commands leading to
unsatisfactory responses, and limited presentation flexibility. Our aim is to
explore AI acceptance factors and refine this approach for more controlled
future studies.
Related papers
- Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic
Systems [67.01132165581667]
We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components.
We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
arXiv Detail & Related papers (2023-11-13T21:20:17Z) - Exploration with Principles for Diverse AI Supervision [88.61687950039662]
Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI.
While this generative AI approach has produced impressive results, it heavily leans on human supervision.
This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation.
We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
arXiv Detail & Related papers (2023-10-13T07:03:39Z) - Beyond XAI:Obstacles Towards Responsible AI [0.0]
Methods of explainability and their evaluation strategies present numerous limitations in real-world contexts.
In this paper, we explore these limitations and discuss their implications in a boarder context of responsible AI.
arXiv Detail & Related papers (2023-09-07T11:08:14Z) - Assigning AI: Seven Approaches for Students, with Prompts [0.0]
This paper examines the transformative role of Large Language Models (LLMs) in education and their potential as learning tools.
The authors propose seven approaches for utilizing AI in classrooms: AI-tutor, AI-coach, AI-mentor, AI-teammate, AI-tool, AI-simulator, and AI-student.
arXiv Detail & Related papers (2023-06-13T03:36:36Z) - Voluminous yet Vacuous? Semantic Capital in an Age of Large Language
Models [0.0]
Large Language Models (LLMs) have emerged as transformative forces in the realm of natural language processing, wielding the power to generate human-like text.
This paper explores the evolution, capabilities, and limitations of these models, while highlighting ethical concerns they raise.
arXiv Detail & Related papers (2023-05-29T09:26:28Z) - Principle-Driven Self-Alignment of Language Models from Scratch with
Minimal Human Supervision [84.31474052176343]
Recent AI-assistant agents, such as ChatGPT, rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback to align the output with human intentions.
This dependence can significantly constrain the true potential of AI-assistant agents due to the high cost of obtaining human supervision.
We propose a novel approach called SELF-ALIGN, which combines principle-driven reasoning and the generative power of LLMs for the self-alignment of AI agents with minimal human supervision.
arXiv Detail & Related papers (2023-05-04T17:59:28Z) - The Role of AI in Drug Discovery: Challenges, Opportunities, and
Strategies [97.5153823429076]
The benefits, challenges and drawbacks of AI in this field are reviewed.
The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
arXiv Detail & Related papers (2022-12-08T23:23:39Z) - Blessing from Human-AI Interaction: Super Reinforcement Learning in
Confounded Environments [19.944163846660498]
We introduce the paradigm of super reinforcement learning that takes advantage of Human-AI interaction for data driven sequential decision making.
In the decision process with unmeasured confounding, the actions taken by past agents can offer valuable insights into undisclosed information.
We develop several super-policy learning algorithms and systematically study their theoretical properties.
arXiv Detail & Related papers (2022-09-29T16:03:07Z) - Human Decision Makings on Curriculum Reinforcement Learning with
Difficulty Adjustment [52.07473934146584]
We guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process.
Our system is highly parallelizable, making it possible for a human to train large-scale reinforcement learning applications.
It shows reinforcement learning performance can successfully adjust in sync with the human desired difficulty level.
arXiv Detail & Related papers (2022-08-04T23:53:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.