Structured AI Demonstrations and Student LLM Use in Engineering Mechanics: Study Design and Preliminary Results

• URL: http://arxiv.org/abs/2607.28710v1
• Date: Thu, 30 Jul 2026 16:38:51 GMT
• Title: Structured AI Demonstrations and Student LLM Use in Engineering Mechanics: Study Design and Preliminary Results
• Abstract summary: The rapid integration of large language models (LLMs) into undergraduate education presents an urgent challenge for engineering instructors.<n>Despite widespread student adoption, there remains a critical lack of domain-specific empirical evidence to guide pedagogical policies.<n>This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026.
• Score: 0.29316801942271303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid integration of large language models (LLMs) into undergraduate education presents an urgent challenge for engineering instructors. Despite widespread student adoption, there remains a critical lack of domain-specific empirical evidence to guide pedagogical policies and classroom interventions. This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026. We detail a reproducible survey instrument used to capture student AI usage patterns, attitudes, and verification practices, which are subsequently linked to academic performance metrics. Additionally, we document a deployable sequence of nine structured, instructor-led AI demonstrations designed to model strategic LLM delegation and evaluation. While our preliminary data highlight shifting student behaviors and complex relationships between AI reliance and course outcomes, the primary contribution of this work is the provision of an open-access methodological framework. By making our complete study design, survey tools, and demonstration materials publicly available, we urge other engineering educators to collect and share similar empirical data. Navigating this unprecedented technological shift will require a collaborative, evidence-based approach to fully understand its long-term impacts on student learning.

Related papers

• How Software Engineering Students Use LLMs to Write Research Papers: An Experience Report [3.415373522515852]
  Large language models are increasingly becoming part of software engineering education.<n>This paper reports an educational experience involving the integration of reflective LLM use into an empirical methods assignment.
  arXiv  Detail & Related papers  (2026-06-03T17:19:43Z)
• Towards Agentic Intelligence for Materials Science [73.4576385477731]
  This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining to goal-conditioned agents interfacing with simulation and experimental platforms.<n>To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science.
  arXiv  Detail & Related papers  (2026-01-29T23:48:43Z)
• Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey [59.3507264893654]
  Issue resolution is a complex Software Engineering task integral to real-world development.<n> benchmarks like SWE-bench revealed this task as profoundly difficult for large language models.<n>This paper presents a systematic survey of this emerging domain.
  arXiv  Detail & Related papers  (2026-01-15T18:55:03Z)
• Vision Generalist Model: A Survey [87.49797517847132]
  We provide a comprehensive overview of the vision generalist models, delving into their characteristics and capabilities within the field.<n>We take a brief excursion into related domains, shedding light on their interconnections and potential synergies.
  arXiv  Detail & Related papers  (2025-06-11T17:23:41Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• The Prompt Canvas: A Literature-Based Practitioner Guide for Creating Effective Prompts in Large Language Models [0.0]
  This paper argues for the creation of an overarching framework that synthesizes existing methodologies into a cohesive overview for practitioners.<n>We present the Prompt Canvas, a structured framework resulting from an extensive literature review on prompt engineering.
  arXiv  Detail & Related papers  (2024-12-06T15:35:18Z)
• O1 Replication Journey: A Strategic Progress Report -- Part 1 [52.062216849476776]
  This paper introduces a pioneering approach to artificial intelligence research, embodied in our O1 Replication Journey. Our methodology addresses critical challenges in modern AI research, including the insularity of prolonged team-based projects. We propose the journey learning paradigm, which encourages models to learn not just shortcuts, but the complete exploration process.
  arXiv  Detail & Related papers  (2024-10-08T15:13:01Z)
• Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study [36.48421439947282]
  This study designed and implemented a prompt engineering intervention at a university in Hong Kong. It examined students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts.
  arXiv  Detail & Related papers  (2024-07-30T15:05:24Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of supervised fine-tuning (SFT)<n>We also review the potential pitfalls of SFT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)
• CLGT: A Graph Transformer for Student Performance Prediction in Collaborative Learning [6.140954034246379]
  We present an extended graph transformer framework for collaborative learning (CLGT) for evaluating and predicting the performance of students. The experimental results confirm that the proposed CLGT outperforms the baseline models in terms of performing predictions based on the real-world datasets.
  arXiv  Detail & Related papers  (2023-07-30T09:54:30Z)
• Demonstrating REACT: a Real-time Educational AI-powered Classroom Tool [0.9899017174990579]
  We present a new Real-time Educational AI-powered Classroom Tool that employs EDM techniques for supporting the decision-making process of educators. ReACT is a data-driven tool with a user-friendly graphical interface. It analyzes students' performance data and provides context-based alerts as well as recommendations to educators for course planning.
  arXiv  Detail & Related papers  (2021-07-30T03:09:59Z)
• A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis [73.74885246830611]
  We propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network (DreGcn) Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-04T14:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.