Pilot Study on Generative AI and Critical Thinking in Higher Education Classrooms

• URL: http://arxiv.org/abs/2509.00167v3
• Date: Mon, 08 Sep 2025 18:37:35 GMT
• Title: Pilot Study on Generative AI and Critical Thinking in Higher Education Classrooms
• Authors: W. F. Lamberti, S. R. Lawrence, D. White, S. Kim, S. Abdullah,
• Abstract summary: Generative AI (GAI) tools have seen rapid adoption in educational settings, yet their role in fostering critical thinking remains underexplored.<n>This pilot study investigates students' ability to apply structured critical thinking when assessing Generative AI outputs in introductory Computational and Data Science courses.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI (GAI) tools have seen rapid adoption in educational settings, yet their role in fostering critical thinking remains underexplored. While previous studies have examined GAI as a tutor for specific lessons or as a tool for completing assignments, few have addressed how students critically evaluate the accuracy and appropriateness of GAI-generated responses. This pilot study investigates students' ability to apply structured critical thinking when assessing Generative AI outputs in introductory Computational and Data Science courses. Given that GAI tools often produce contextually flawed or factually incorrect answers, we designed learning activities that require students to analyze, critique, and revise AI-generated solutions. Our findings offer initial insights into students' ability to engage critically with GAI content and lay the groundwork for more comprehensive studies in future semesters.

Related papers

• The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research [56.80927148740585]
  We address the challenges of scalability and rigor by flipping the dynamic and developing AI agents as research evaluators.<n>We use mechanistic interpretability research as a testbed, build standardized research output, and develop MechEvalAgent.<n>Our work demonstrates the potential of AI agents to transform research evaluation and pave the way for rigorous scientific practices.
  arXiv  Detail & Related papers  (2026-02-05T19:00:02Z)
• FML-bench: A Benchmark for Automatic ML Research Agents Highlighting the Importance of Exploration Breadth [43.606494515048524]
  Large language models (LLMs) have sparked growing interest in automatic machine learning research agents.<n>Existing benchmarks tend to overemphasize engineering aspects while neglecting academic rigor.<n>We introduce FML-bench, a benchmark designed to evaluate automatic machine learning research agents on 8 diverse and fundamental machine learning research problems.
  arXiv  Detail & Related papers  (2025-10-12T06:41:05Z)
• Predicting ChatGPT Use in Assignments: Implications for AI-Aware Assessment Design [0.0]
  The study reveals that frequent use of ChatGPT for learning new concepts correlates with potential overreliance.<n>We propose discipline-specific guidelines and reimagined assessment strategies to balance innovation with academic rigor.
  arXiv  Detail & Related papers  (2025-08-16T11:09:38Z)
• Assessing the Quality of AI-Generated Exams: A Large-Scale Field Study [18.104664166381877]
  Large language models (LLMs) challenge conventional methods of teaching and learning.<n>One promising application is the generation of customized exams, tailored to specific course content.
  arXiv  Detail & Related papers  (2025-08-09T01:20:53Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Beyond Detection: Designing AI-Resilient Assessments with Automated Feedback Tool to Foster Critical Thinking [0.0]
  This research proposes a proactive, AI-resilient solution based on assessment design rather than detection.<n>It introduces a web-based Python tool that integrates Bloom's taxonomy with advanced natural language processing techniques.<n>It helps educators determine whether a task targets lower-order thinking such as recall and summarization or higher-order skills such as analysis, evaluation, and creation.
  arXiv  Detail & Related papers  (2025-03-30T23:13:00Z)
• Analyzing the Impact of AI Tools on Student Study Habits and Academic Performance [0.0]
  The research focuses on how AI tools can support personalized learning, adaptive test adjustments, and provide real-time classroom analysis.<n>Student feedback revealed strong support for these features, and the study found a significant reduction in study hours alongside an increase in GPA.<n>Despite these benefits, challenges such as over-reliance on AI and difficulties in integrating AI with traditional teaching methods were also identified.
  arXiv  Detail & Related papers  (2024-12-03T04:51:57Z)
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants [176.39275404745098]
  We evaluate whether two AI assistants, GPT-3.5 and GPT-4, can adequately answer assessment questions.<n>GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions.<n>Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
  arXiv  Detail & Related papers  (2024-08-07T12:11:49Z)
• The Potential and Implications of Generative AI on HCI Education [10.557784268438779]
  Generative AI (GAI) is impacting teaching and learning directly or indirectly across a range of subjects and disciplines. We report on the main pedagogical insights gained from the inclusion of generative AI into a 10 week undergraduate module.
  arXiv  Detail & Related papers  (2024-05-08T15:46:31Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Towards Goal-oriented Intelligent Tutoring Systems in Online Education [65.17234980710386]
  We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS)<n>GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.<n>We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
  arXiv  Detail & Related papers  (2023-12-03T12:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.