Towards a Success Model for Automated Programming Assessment Systems
Used as a Formative Assessment Tool
- URL: http://arxiv.org/abs/2306.04958v1
- Date: Thu, 8 Jun 2023 06:19:15 GMT
- Title: Towards a Success Model for Automated Programming Assessment Systems
Used as a Formative Assessment Tool
- Authors: Clemens Sauerwein, Tobias Antensteiner, Stefan Oppl, Iris Groher,
Alexander Meschtscherjakov, Philipp Zech and Ruth Breu
- Abstract summary: The assessment of source code in university education is a central and important task for lecturers of programming courses.
The use of automated programming assessment systems (APASs) is a promising solution.
Measuring the effectiveness and success of APASs is crucial to understanding how such platforms should be designed, implemented, and used.
- Score: 42.03652286907358
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The assessment of source code in university education is a central and
important task for lecturers of programming courses. In doing so, educators are
confronted with growing numbers of students having increasingly diverse
prerequisites, a shortage of tutors, and highly dynamic learning objectives. To
support lecturers in meeting these challenges, the use of automated programming
assessment systems (APASs), facilitating formative assessments by providing
timely, objective feedback, is a promising solution. Measuring the
effectiveness and success of these platforms is crucial to understanding how
such platforms should be designed, implemented, and used. However, research and
practice lack a common understanding of aspects influencing the success of
APASs. To address these issues, we have devised a success model for APASs based
on established models from information systems as well as blended learning
research and conducted an online survey with 414 students using the same APAS.
In addition, we examined the role of mediators intervening between technology-,
system- or self-related factors, respectively, and the users' satisfaction with
APASs. Ultimately, our research has yielded a model of success comprising seven
constructs influencing user satisfaction with an APAS.
Related papers
- Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset [94.13848736705575]
We introduce Facial Identity Unlearning Benchmark (FIUBench), a novel VLM unlearning benchmark designed to robustly evaluate the effectiveness of unlearning algorithms.
We apply a two-stage evaluation pipeline that is designed to precisely control the sources of information and their exposure levels.
Through the evaluation of four baseline VLM unlearning algorithms within FIUBench, we find that all methods remain limited in their unlearning performance.
arXiv Detail & Related papers (2024-11-05T23:26:10Z) - Personalised Feedback Framework for Online Education Programmes Using Generative AI [0.0]
This paper presents an alternative feedback framework which extends the capabilities of ChatGPT by integrating embeddings.
As part of the study, we proposed and developed a proof of concept solution, achieving an efficacy rate of 90% and 100% for open-ended and multiple-choice questions.
arXiv Detail & Related papers (2024-10-14T22:35:40Z) - Identifying Student Profiles Within Online Judge Systems Using
Explainable Artificial Intelligence [6.638206014723678]
Online Judge (OJ) systems are typically considered within programming-related courses as they yield fast and objective assessments of the code developed by the students.
This work aims to tackle this limitation by considering the further exploitation of the information gathered by the OJ and automatically inferring feedback for both the student and the instructor.
arXiv Detail & Related papers (2024-01-29T12:11:30Z) - Lessons Learned from Designing an Open-Source Automated Feedback System
for STEM Education [5.326069675013602]
We present RATsApp, an open-source automated feedback system (AFS) that incorporates research-based features such as formative feedback.
The system focuses on core STEM competencies such as mathematical competence, representational competence, and data literacy.
As an open-source platform, RATsApp encourages public contributions to its ongoing development, fostering a collaborative approach to improve educational tools.
arXiv Detail & Related papers (2024-01-19T07:13:07Z) - Towards Goal-oriented Intelligent Tutoring Systems in Online Education [69.06930979754627]
We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS)
GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.
We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
arXiv Detail & Related papers (2023-12-03T12:37:16Z) - UKP-SQuARE: An Interactive Tool for Teaching Question Answering [61.93372227117229]
The exponential growth of question answering (QA) has made it an indispensable topic in any Natural Language Processing (NLP) course.
We introduce UKP-SQuARE as a platform for QA education.
Students can run, compare, and analyze various QA models from different perspectives.
arXiv Detail & Related papers (2023-05-31T11:29:04Z) - Lifelong Learning Metrics [63.8376359764052]
The DARPA Lifelong Learning Machines (L2M) program seeks to yield advances in artificial intelligence (AI) systems.
This document outlines a formalism for constructing and characterizing the performance of agents performing lifelong learning scenarios.
arXiv Detail & Related papers (2022-01-20T16:29:14Z) - Application of Deep Self-Attention in Knowledge Tracing [2.5852720579998336]
This paper proposed Deep Self-Attentive Knowledge Tracing (DSAKT) based on the data of PTA, an online assessment system used by students in many universities in China.
Experimentation on the data of PTA shows that DSAKT outperforms the other models for knowledge tracing an improvement of 2.1% on average.
arXiv Detail & Related papers (2021-05-17T14:45:38Z) - Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal.
In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
arXiv Detail & Related papers (2021-01-19T12:23:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.