From Co-Design to Metacognitive Laziness: Evaluating Generative AI in Vocational Education
- URL: http://arxiv.org/abs/2512.12306v1
- Date: Sat, 13 Dec 2025 12:26:25 GMT
- Title: From Co-Design to Metacognitive Laziness: Evaluating Generative AI in Vocational Education
- Authors: Amir Yunus, Peng Rend Gay, Oon Teng Lee,
- Abstract summary: This study examines the development and deployment of a Generative AI proof-of-concept (POC) designed to support lecturers in a vocational education setting in Singapore.<n>The POC achieved its primary operational goals: lecturers reported streamlined, reduced cognitive load, and observed improved student confidence in navigating course content.<n>Despite enhanced teaching processes, performance data revealed no significant improvement in overall student assessment outcomes.<n>The study raises consequential design questions regarding how AI tools can be engineered to minimise dependency, metacognitive development, and calibrate support across varying ability levels.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This study examines the development and deployment of a Generative AI proof-of-concept (POC) designed to support lecturers in a vocational education setting in Singapore. Employing a user-centred, mixed-methods design process, we co-developed an AI chatbot with lecturers to address recurring instructional challenges during exam preparation, specifically managing repetitive questions and scaling feedback delivery. The POC achieved its primary operational goals: lecturers reported streamlined workflows, reduced cognitive load, and observed improved student confidence in navigating course content. However, the deployment yielded unexpected insights into student learning behaviours. Despite enhanced teaching processes, performance data revealed no significant improvement in overall student assessment outcomes. Deep analysis of interaction logs identified concerning patterns, including self-efficacy-driven dependency, "metacognitive laziness" (cognitive offloading), and divergent usage strategies. While high-ability students leveraged the tool for strategic verification, low-ability students frequently used it to bypass cognitive effort, potentially exacerbating performance gaps. These findings suggest that Generative AI's educational influence extends beyond instructional efficiency to shape cognitive engagement, self-regulation, and learner equity. The study raises consequential design questions regarding how AI tools can be engineered to minimise dependency, scaffold metacognitive development, and calibrate support across varying ability levels. We conclude that while Generative AI can substantially enhance the teaching experience, achieving meaningful learning gains requires rigorous attention to learner behaviour and the equitable design of AI-supported environments.
Related papers
- AI Combines, Humans Socialise: A SECI-based Experience Report on Business Simulation Games [0.0]
This paper reports on the integration of generative AI tools into a Business Simulation Games (BSG) designed for engineering students.<n> AI was embedded as a support mechanism during the simulation to assist students in analysing events, reformulating information, and generating decision-relevant insights.<n>The results suggest a functional boundary in human-AI collaboration within simulation-based learning.
arXiv Detail & Related papers (2026-02-24T07:26:16Z) - The Imperfect Learner: Incorporating Developmental Trajectories in Memory-based Student Simulation [55.722188569369656]
This paper introduces a novel framework for memory-based student simulation.<n>It incorporates developmental trajectories through a hierarchical memory mechanism with structured knowledge representation.<n>In practice, we implement a curriculum-aligned simulator grounded on the Next Generation Science Standards.
arXiv Detail & Related papers (2025-11-08T08:05:43Z) - Socratic Mind: Impact of a Novel GenAI-Powered Assessment Tool on Student Learning and Higher-Order Thinking [2.192176712066146]
This study examines the impact of Socratic Mind, a Generative Artificial Intelligence (GenAI) powered formative assessment tool on learning outcomes.<n>Students who engaged with the GenAI tool experienced significant gains in their quiz scores compared to those who did not.<n>Our findings highlight the promise of AI-mediated dialogue in fostering deeper engagement and higher-order cognitive skills.
arXiv Detail & Related papers (2025-09-18T03:08:24Z) - AI-Educational Development Loop (AI-EDL): A Conceptual Framework to Bridge AI Capabilities with Classical Educational Theories [8.500617875591633]
This study introduces the AI-Educational Development Loop (AI-EDL), a theory-driven framework that integrates classical learning theories with human-in-the-loop artificial intelligence (AI)<n>The framework emphasizes transparency, self-regulated learning, and pedagogical oversight.
arXiv Detail & Related papers (2025-08-01T15:44:19Z) - ChatGPT produces more "lazy" thinkers: Evidence of cognitive engagement decline [0.0]
This study investigates the impact of generative artificial intelligence (AI) tools on the cognitive engagement of students during academic writing tasks.<n>The results revealed significantly lower cognitive engagement scores in the ChatGPT group compared to the control group.<n>These findings suggest that AI assistance may lead to cognitive offloading.
arXiv Detail & Related papers (2025-06-30T18:41:50Z) - When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
arXiv Detail & Related papers (2025-06-05T20:48:16Z) - Dynamic Programming Techniques for Enhancing Cognitive Representation in Knowledge Tracing [125.75923987618977]
We propose the Cognitive Representation Dynamic Programming based Knowledge Tracing (CRDP-KT) model.<n>It is a dynamic programming algorithm to optimize cognitive representations based on the difficulty of the questions and the performance intervals between them.<n>It provides more accurate and systematic input features for subsequent model training, thereby minimizing distortion in the simulation of cognitive states.
arXiv Detail & Related papers (2025-06-03T14:44:48Z) - Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training [86.70255651945602]
We introduce a novel inference-time steering methodology called Reinforcing Cognitive Experts (RICE)<n>RICE aims to improve reasoning performance without additional training or complexs.<n> Empirical evaluations with leading MoE-based LRMs demonstrate noticeable and consistent improvements in reasoning accuracy, cognitive efficiency, and cross-domain generalization.
arXiv Detail & Related papers (2025-05-20T17:59:16Z) - Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
arXiv Detail & Related papers (2025-02-18T17:12:26Z) - Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance [0.0]
The research focuses on how AI tools can support personalized learning, adaptive test adjustments, and provide real-time classroom analysis.<n>Student feedback revealed strong support for these features, and the study found a significant reduction in study hours alongside an increase in GPA.<n>Despite these benefits, challenges such as over-reliance on AI and difficulties in integrating AI with traditional teaching methods were also identified.
arXiv Detail & Related papers (2024-12-03T04:51:57Z) - Towards Goal-oriented Intelligent Tutoring Systems in Online Education [65.17234980710386]
We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS)<n>GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.<n>We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
arXiv Detail & Related papers (2023-12-03T12:37:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.