Impact of AI Tools on Learning Outcomes: Decreasing Knowledge and Over-Reliance

• URL: http://arxiv.org/abs/2510.16019v1
• Date: Wed, 15 Oct 2025 13:48:14 GMT
• Title: Impact of AI Tools on Learning Outcomes: Decreasing Knowledge and Over-Reliance
• Abstract summary: Students at all levels of education are increasingly relying on generative artificial intelligence (AI) tools to complete assignments and achieve higher exam scores.<n>To investigate the impact of generative AI on learning outcomes, an experiment was conducted at Corvinus University of Budapest.<n>Despite the organizers' best efforts to explain the design and to create equal opportunities for all participants, many students perceived the experiment as a major disruption.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Students at all levels of education are increasingly relying on generative artificial intelligence (AI) tools to complete assignments and achieve higher exam scores. However, it remains unclear how this reliance affects their motivation, their genuine understanding of the material, and the extent to which it substitutes for the process of knowledge acquisition. To investigate the impact of generative AI on learning outcomes, an experiment was conducted at Corvinus University of Budapest. In an operations research class, students were randomly assigned into two groups: one was permitted to use AI tools during classes and examinations, while the other was not. To ensure fairness, a compensation mechanism was introduced: students in the lower-performing group received point adjustments until the average performance of the two groups was equalized. Despite the organizers' best efforts to explain the design and to create equal opportunities for all participants, many students perceived the experiment as a major disruption. Although the experiment was approved by every relevant university authority -- including the Ethics Board, the Head of Department, the Program Director, and the Student Council -- students escalated their concerns to the media and eventually to the State Secretary for Higher Education of Hungary. As a result, the experiment had to be substantially revised before completion: on the final exam the test group was merged with the control group. Still, the data allowed us to draw decisive conclusions regarding the students' learning habits. Uncontrolled use of AI tools leads to disengaged students and low understanding of material. The extreme reactions of the students proved even more revealing than the data collected: generative AI tools have already become indispensable for students, raising fundamental questions about the validity of their learning process.

Related papers

• AI literacy over tool design: a mixed-methods study of scaffolded versus unrestricted generative AI in programming education [0.0]
  This paper reports a seven-week mixed-methods pilot study in a master's-level data analytics course.<n>33 students were randomly assigned either to a scaffolded AI Study Coach embedded in the notebook-based laboratory sessions or to unrestricted use of AI tools of their own choosing.<n>The design of the tool mattered less than the students' capacity to govern their own use of it, a capacity that is at present acquired by chance.
  arXiv  Detail & Related papers  (2026-09-15T07:53:58Z)
• Is Solving Better Than Evaluating GenAI Solutions? [45.714140455691485]
  This study compares evaluating GenAI-generated solutions with traditional problem solving in a junior-level algorithms course.<n>We found no statistically significant differences between groups in midterm scores, final exam scores, overall course grades, or exam problems structurally aligned with the homework interventions.<n>Students received significantly higher homework scores when evaluating GenAI-generated solutions, but this localized advantage did not translate into downstream gains.
  arXiv  Detail & Related papers  (2026-07-30T02:08:29Z)
• Beyond Detection: Redesigning Assessment and Governande of Generative AI at the Universidad Politécnica de Madrid (UPM) [39.92037610044293]
  Most institutions frame the debate around AI detection, plagiarism, academic integrity and a presumed drop in student effort as defensive.<n>Other group of pioneering universities is pursuing deeper adoption, and assuming that any policy built on prevention or sanction will not hold.<n>What universities need instead is a coordinated effort to set clear, course-by-course rules for GenAI use.<n>UPM is developing a strategic and sustainable AI policy and adoption framework structured around six dimensions.
  arXiv  Detail & Related papers  (2026-06-04T18:04:50Z)
• Framing, Judging, Steering: An Assessable Competency Model for Teach-ing Students to Reason With Generative AI [47.027290803102666]
  We propose CoRe-3 (Co-Reasoning), a competency model factoring productive AI use into three assessable skills.<n>We ground the skills in theory, state five testable propositions, and instantiate them in CoReasoningLab.<n>Human-rater agreement and outcomes are next; we release the instrument, data, and protocol.
  arXiv  Detail & Related papers  (2026-06-04T10:25:31Z)
• Teaching Students to Question the Machine: An AI Literacy Intervention Improves Students' Regulation of LLM Use in a Science Task [0.0]
  The rapid adoption of generative artificial intelligence in schools raises concerns about students' uncritical reliance on its outputs.<n>These skills are still developing in middle school, making students vulnerable to over-trust and premature acceptance of AI outputs.<n>We report a controlled classroom study examining whether a two-hour AI literacy workshop improves students' interaction strategies and quality of final answers.
  arXiv  Detail & Related papers  (2026-04-02T12:19:43Z)
• AI Literacy, Safety Awareness, and STEM Career Aspirations of Australian Secondary Students: Evaluating the Impact of Workshop Interventions [38.350232667249095]
  Deepfakes and other forms of synthetic media pose growing safety risks for adolescents.<n>This study evaluates the impact of Day of AI Australia's workshop-based intervention on Australian secondary students.
  arXiv  Detail & Related papers  (2026-01-30T02:55:53Z)
• Socratic Mind: Impact of a Novel GenAI-Powered Assessment Tool on Student Learning and Higher-Order Thinking [2.192176712066146]
  This study examines the impact of Socratic Mind, a Generative Artificial Intelligence (GenAI) powered formative assessment tool on learning outcomes.<n>Students who engaged with the GenAI tool experienced significant gains in their quiz scores compared to those who did not.<n>Our findings highlight the promise of AI-mediated dialogue in fostering deeper engagement and higher-order cognitive skills.
  arXiv  Detail & Related papers  (2025-09-18T03:08:24Z)
• Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study [10.71612026319996]
  This study analyzed 315 student-AI conversations during a brief, quiz-based scenario across various STEM courses.<n>Students exhibited overall low reliance on AI and many of them could not effectively use AI for learning.<n>Certain behavioral metrics strongly predicted AI reliance, highlighting potential behavioral mechanisms to explain AI adoption.
  arXiv  Detail & Related papers  (2025-08-27T20:00:27Z)
• Short-Term Gains, Long-Term Gaps: The Impact of GenAI and Search Technologies on Retention [1.534667887016089]
  This study investigates how GenAI (ChatGPT), search engines (Google), and e-textbooks influence student performance across tasks of varying cognitive complexity.<n>ChatGPT and Google groups outperformed the control group in immediate assessments for lower-order cognitive tasks.<n>While AI-driven tools facilitate immediate performance, they do not inherently reinforce long-term retention unless supported by structured learning strategies.
  arXiv  Detail & Related papers  (2025-07-10T00:44:50Z)
• AI instructional agent improves student's perceived learner control and learning outcome: empirical evidence from a randomized controlled trial [29.45751702212421]
  This study examines the impact of an AI instructional agent on students' perceived learner control and academic performance in a medium demanding course with lecturing as the main teaching strategy.<n>Students in the AI instructional agent group reported significantly higher levels of perceived learner control compared to the other groups.
  arXiv  Detail & Related papers  (2025-05-28T16:13:27Z)
• Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses [0.0]
  Generative AI (GenAI) is rapidly entering computer science education.<n>Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses.<n>This study examines the impact of a dedicated "AI-Lab" intervention on undergraduate students.
  arXiv  Detail & Related papers  (2025-04-30T18:12:42Z)
• Let people fail! Exploring the influence of explainable virtual and robotic agents in learning-by-doing tasks [45.23431596135002]
  This study compares the effects of classic vs. partner-aware explanations on human behavior and performance during a learning-by-doing task. Results indicated that partner-aware explanations influenced participants differently based on the type of artificial agents involved.
  arXiv  Detail & Related papers  (2024-11-15T13:22:04Z)
• Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study [36.48421439947282]
  This study designed and implemented a prompt engineering intervention at a university in Hong Kong. It examined students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts.
  arXiv  Detail & Related papers  (2024-07-30T15:05:24Z)
• Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students [35.05435052195561]
  The increasing use of Artificial Intelligence (AI) by students in learning presents new challenges for assessing their learning outcomes. This paper introduces a co-design study to explore the potential of students' AI usage data as a novel material for assessment.
  arXiv  Detail & Related papers  (2024-01-26T14:49:29Z)
• Learning to Prompt in the Classroom to Understand AI Limits: A pilot study [35.06607166918901]
  Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems. However, excitement has led to negative sentiments, even as AI methods demonstrate remarkable contributions. A pilot educational intervention was performed in a high school with 21 students.
  arXiv  Detail & Related papers  (2023-07-04T07:51:37Z)
• Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task [32.6507926764587]
  Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software. We show that deep reinforcement learning can be used to provide adaptive pedagogical support to students learning about the concept of volume.
  arXiv  Detail & Related papers  (2023-04-11T02:11:24Z)
• Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble [56.705249154629264]
  Self-training teacher-student frameworks are proposed to improve the robustness of NER models. In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks. Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
  arXiv  Detail & Related papers  (2022-12-13T12:14:09Z)
• Disadvantaged students increase their academic performance through collective intelligence exposure in emergency remote learning due to COVID 19 [105.54048699217668]
  During the COVID-19 crisis, educational institutions worldwide shifted from face-to-face instruction to emergency remote teaching (ERT) modalities. We analyzed data on 7,528 undergraduate students and found that cooperative and consensus dynamics among students in discussion forums positively affect their final GPA. Using natural language processing, we show that first-year students with low academic performance during high school are exposed to more content-intensive posts in discussion forums.
  arXiv  Detail & Related papers  (2022-03-10T20:23:38Z)
• Collaborative Group Learning [42.31194030839819]
  Collaborative learning has successfully applied knowledge transfer to guide a pool of small student networks towards robust local minima. Previous approaches typically struggle with drastically aggravated student homogenization when the number of students rises. We propose Collaborative Group Learning, an efficient framework that aims to diversify the feature representation and conduct an effective regularization.
  arXiv  Detail & Related papers  (2020-09-16T14:34:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.