AI Mentors for Student Projects: Spotting Early Issues in Computer Science Proposals

• URL: http://arxiv.org/abs/2503.05782v1
• Date: Wed, 26 Feb 2025 16:24:14 GMT
• Title: AI Mentors for Student Projects: Spotting Early Issues in Computer Science Proposals
• Authors: Gati Aher, Robin Schmucker, Tom Mitchell, Zachary C. Lipton,
• Abstract summary: We develop a software system that collects project proposals and information to support educators in determining whether a student is ready to engage with aptitude.<n>We find that users perceived the system as helpful for writing project proposals and identifying tools and technologies to learn more about.<n>While the prospect of using LLMs to rate the quality of students' project proposals is promising, its long-term effectiveness hinges on future efforts at characterizing indicators that reliably predict students' success and motivation to learn.
• Score: 35.92015790655676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When executed well, project-based learning (PBL) engages students' intrinsic motivation, encourages students to learn far beyond a course's limited curriculum, and prepares students to think critically and maturely about the skills and tools at their disposal. However, educators experience mixed results when using PBL in their classrooms: some students thrive with minimal guidance and others flounder. Early evaluation of project proposals could help educators determine which students need more support, yet evaluating project proposals and student aptitude is time-consuming and difficult to scale. In this work, we design, implement, and conduct an initial user study (n = 36) for a software system that collects project proposals and aptitude information to support educators in determining whether a student is ready to engage with PBL. We find that (1) users perceived the system as helpful for writing project proposals and identifying tools and technologies to learn more about, (2) educator ratings indicate that users with less technical experience in the project topic tend to write lower-quality project proposals, and (3) GPT-4o's ratings show agreement with educator ratings. While the prospect of using LLMs to rate the quality of students' project proposals is promising, its long-term effectiveness strongly hinges on future efforts at characterizing indicators that reliably predict students' success and motivation to learn.

Related papers

• Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization [69.96794098855938]
  Weak-to-strong generalization (W2SG) offers a promising framework for supervising increasingly capable language models (LLMs) Traditional W2SG methods rely on passive learning, where a weak teacher provides noisy demonstrations to train a strong student. We introduce Alice, a framework that leverages complementary knowledge between teacher and student to enhance the learning process.
  arXiv  Detail & Related papers  (2025-04-09T22:33:06Z)
• Co-designing Large Language Model Tools for Project-Based Learning with K12 Educators [10.100127895043235]
  generative large language models (LLMs) have opened the door for student-based active learning methods.<n>Project design and management, assessment and guidance are challenges for student implementation.<n>We propose guidelines for future deployment of generative large language models in classrooms.
  arXiv  Detail & Related papers  (2025-02-13T22:23:08Z)
• Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students [53.20318273452059]
  Large language models (LLMs) like OpenAI's ChatGPT have opened up new avenues in education.<n>Despite school restrictions, our survey of over 300 middle and high school students revealed that a remarkable 70% of students have utilized LLMs.<n>We propose a few ideas to address such issues, including subject-specific models, personalized learning, and AI classrooms.
  arXiv  Detail & Related papers  (2024-11-27T19:19:34Z)
• "Give me the code" -- Log Analysis of First-Year CS Students' Interactions With GPT [0.0]
  This paper analyzes the prompts used by 69 freshmen undergraduate students to solve a certain programming problem within a project assignment.<n>Despite using unsophisticated prompting techniques, our findings suggest that the majority of students successfully leveraged GPT.<n>Half of the students demonstrated the ability to exercise judgment in selecting from multiple GPT-generated solutions.
  arXiv  Detail & Related papers  (2024-11-26T20:11:46Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring. We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue. We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Explainable Few-shot Knowledge Tracing [48.877979333221326]
  We propose a cognition-guided framework that can track the student knowledge from a few student records while providing natural language explanations. Experimental results from three widely used datasets show that LLMs can perform comparable or superior to competitive deep knowledge tracing methods.
  arXiv  Detail & Related papers  (2024-05-23T10:07:21Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• Identifying Student Profiles Within Online Judge Systems Using Explainable Artificial Intelligence [6.638206014723678]
  Online Judge (OJ) systems are typically considered within programming-related courses as they yield fast and objective assessments of the code developed by the students. This work aims to tackle this limitation by considering the further exploitation of the information gathered by the OJ and automatically inferring feedback for both the student and the instructor.
  arXiv  Detail & Related papers  (2024-01-29T12:11:30Z)
• A Machine Learning system to monitor student progress in educational institutes [0.0]
  We propose a data driven approach that makes use of Machine Learning techniques to generate a classifier called credit score. The proposal to use credit score as progress indicator is well suited to be used in a Learning Management System.
  arXiv  Detail & Related papers  (2022-11-02T08:24:08Z)
• Experience of Teaching Data Visualization using Project-based Learning [0.3437656066916039]
  We show which input was provided when necessary for students to achieve their goals. We discuss and compare the tools we found useful for students to accomplish their goals.
  arXiv  Detail & Related papers  (2021-10-21T16:47:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.