Lessons Learned from Designing an Open-Source Automated Feedback System
for STEM Education
- URL: http://arxiv.org/abs/2401.10531v1
- Date: Fri, 19 Jan 2024 07:13:07 GMT
- Title: Lessons Learned from Designing an Open-Source Automated Feedback System
for STEM Education
- Authors: Steffen Steinert, Lars Krupp, Karina E. Avila, Anke S. Janssen, Verena
Ruf, David Dzsotjan, Christian De Schryver, Jakob Karolus, Stefan Ruzika,
Karen Joisten, Paul Lukowicz, Jochen Kuhn, Norbert Wehn, Stefan K\"uchemann
- Abstract summary: We present RATsApp, an open-source automated feedback system (AFS) that incorporates research-based features such as formative feedback.
The system focuses on core STEM competencies such as mathematical competence, representational competence, and data literacy.
As an open-source platform, RATsApp encourages public contributions to its ongoing development, fostering a collaborative approach to improve educational tools.
- Score: 5.326069675013602
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: As distance learning becomes increasingly important and artificial
intelligence tools continue to advance, automated systems for individual
learning have attracted significant attention. However, the scarcity of
open-source online tools that are capable of providing personalized feedback
has restricted the widespread implementation of research-based feedback
systems. In this work, we present RATsApp, an open-source automated feedback
system (AFS) that incorporates research-based features such as formative
feedback. The system focuses on core STEM competencies such as mathematical
competence, representational competence, and data literacy. It also allows
lecturers to monitor students' progress. We conducted a survey based on the
technology acceptance model (TAM2) among a set of students (N=64). Our findings
confirm the applicability of the TAM2 framework, revealing that factors such as
the relevance of the studies, output quality, and ease of use significantly
influence the perceived usefulness. We also found a linear relation between the
perceived usefulness and the intention to use, which in turn is a significant
predictor of the frequency of use. Moreover, the formative feedback feature of
RATsApp received positive feedback, indicating its potential as an educational
tool. Furthermore, as an open-source platform, RATsApp encourages public
contributions to its ongoing development, fostering a collaborative approach to
improve educational tools.
Related papers
- KBAlign: Efficient Self Adaptation on Specific Knowledge Bases [75.78948575957081]
Large language models (LLMs) usually rely on retrieval-augmented generation to exploit knowledge materials in an instant manner.
We propose KBAlign, an approach designed for efficient adaptation to downstream tasks involving knowledge bases.
Our method utilizes iterative training with self-annotated data such as Q&A pairs and revision suggestions, enabling the model to grasp the knowledge content efficiently.
arXiv Detail & Related papers (2024-11-22T08:21:03Z) - Personalised Feedback Framework for Online Education Programmes Using Generative AI [0.0]
This paper presents an alternative feedback framework which extends the capabilities of ChatGPT by integrating embeddings.
As part of the study, we proposed and developed a proof of concept solution, achieving an efficacy rate of 90% and 100% for open-ended and multiple-choice questions.
arXiv Detail & Related papers (2024-10-14T22:35:40Z) - AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment [12.970776782360366]
AERA Chat is an interactive platform to provide visually explained assessment of student answers.
Users can input questions and student answers to obtain automated, explainable assessment results from large language models.
arXiv Detail & Related papers (2024-10-12T11:57:53Z) - Improving the Validity of Automatically Generated Feedback via
Reinforcement Learning [50.067342343957876]
We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)
Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
arXiv Detail & Related papers (2024-03-02T20:25:50Z) - Towards Goal-oriented Intelligent Tutoring Systems in Online Education [69.06930979754627]
We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS)
GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.
We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
arXiv Detail & Related papers (2023-12-03T12:37:16Z) - Empowering Private Tutoring by Chaining Large Language Models [87.76985829144834]
This work explores the development of a full-fledged intelligent tutoring system powered by state-of-the-art large language models (LLMs)
The system is into three inter-connected core processes-interaction, reflection, and reaction.
Each process is implemented by chaining LLM-powered tools along with dynamically updated memory modules.
arXiv Detail & Related papers (2023-09-15T02:42:03Z) - A Domain-Agnostic Approach for Characterization of Lifelong Learning
Systems [128.63953314853327]
"Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability.
We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
arXiv Detail & Related papers (2023-01-18T21:58:54Z) - Application of Deep Self-Attention in Knowledge Tracing [2.5852720579998336]
This paper proposed Deep Self-Attentive Knowledge Tracing (DSAKT) based on the data of PTA, an online assessment system used by students in many universities in China.
Experimentation on the data of PTA shows that DSAKT outperforms the other models for knowledge tracing an improvement of 2.1% on average.
arXiv Detail & Related papers (2021-05-17T14:45:38Z) - Automated Personalized Feedback Improves Learning Gains in an
Intelligent Tutoring System [34.19909376464836]
We investigate how automated, data-driven, personalized feedback in a large-scale intelligent tutoring system (ITS) improves student learning outcomes.
We propose a machine learning approach to generate personalized feedback, which takes individual needs of students into account.
We utilize state-of-the-art machine learning and natural language processing techniques to provide the students with personalized hints, Wikipedia-based explanations, and mathematical hints.
arXiv Detail & Related papers (2020-05-05T18:30:08Z) - Knowledge-guided Deep Reinforcement Learning for Interactive
Recommendation [49.32287384774351]
Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy.
We propose Knowledge-Guided deep Reinforcement learning to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation.
arXiv Detail & Related papers (2020-04-17T05:26:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.