Lessons Learned from Designing an Open-Source Automated Feedback System
for STEM Education
- URL: http://arxiv.org/abs/2401.10531v1
- Date: Fri, 19 Jan 2024 07:13:07 GMT
- Title: Lessons Learned from Designing an Open-Source Automated Feedback System
for STEM Education
- Authors: Steffen Steinert, Lars Krupp, Karina E. Avila, Anke S. Janssen, Verena
Ruf, David Dzsotjan, Christian De Schryver, Jakob Karolus, Stefan Ruzika,
Karen Joisten, Paul Lukowicz, Jochen Kuhn, Norbert Wehn, Stefan K\"uchemann
- Abstract summary: We present RATsApp, an open-source automated feedback system (AFS) that incorporates research-based features such as formative feedback.
The system focuses on core STEM competencies such as mathematical competence, representational competence, and data literacy.
As an open-source platform, RATsApp encourages public contributions to its ongoing development, fostering a collaborative approach to improve educational tools.
- Score: 5.326069675013602
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: As distance learning becomes increasingly important and artificial
intelligence tools continue to advance, automated systems for individual
learning have attracted significant attention. However, the scarcity of
open-source online tools that are capable of providing personalized feedback
has restricted the widespread implementation of research-based feedback
systems. In this work, we present RATsApp, an open-source automated feedback
system (AFS) that incorporates research-based features such as formative
feedback. The system focuses on core STEM competencies such as mathematical
competence, representational competence, and data literacy. It also allows
lecturers to monitor students' progress. We conducted a survey based on the
technology acceptance model (TAM2) among a set of students (N=64). Our findings
confirm the applicability of the TAM2 framework, revealing that factors such as
the relevance of the studies, output quality, and ease of use significantly
influence the perceived usefulness. We also found a linear relation between the
perceived usefulness and the intention to use, which in turn is a significant
predictor of the frequency of use. Moreover, the formative feedback feature of
RATsApp received positive feedback, indicating its potential as an educational
tool. Furthermore, as an open-source platform, RATsApp encourages public
contributions to its ongoing development, fostering a collaborative approach to
improve educational tools.
Related papers
- From Questions to Insightful Answers: Building an Informed Chatbot for University Resources [2.563648444530805]
This paper presents a Large Language Model (LLM)-based system built using Retrieval Augmented Generation (RAG) pipelines.
Our system leverages university data as an external data corpus and ingests it into our RAG pipelines for domain-specific question-answering tasks.
We evaluate the effectiveness of our system in generating accurate and pertinent responses for Mississippi State University, as a case study.
arXiv Detail & Related papers (2024-05-13T19:05:42Z) - Improving the Validity of Automatically Generated Feedback via
Reinforcement Learning [50.067342343957876]
We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)
Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
arXiv Detail & Related papers (2024-03-02T20:25:50Z) - Towards Goal-oriented Intelligent Tutoring Systems in Online Education [69.06930979754627]
We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS)
GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.
We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
arXiv Detail & Related papers (2023-12-03T12:37:16Z) - Empowering Private Tutoring by Chaining Large Language Models [87.76985829144834]
This work explores the development of a full-fledged intelligent tutoring system powered by state-of-the-art large language models (LLMs)
The system is into three inter-connected core processes-interaction, reflection, and reaction.
Each process is implemented by chaining LLM-powered tools along with dynamically updated memory modules.
arXiv Detail & Related papers (2023-09-15T02:42:03Z) - A Domain-Agnostic Approach for Characterization of Lifelong Learning
Systems [128.63953314853327]
"Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability.
We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
arXiv Detail & Related papers (2023-01-18T21:58:54Z) - Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic
Approach [59.77710485234197]
This paper presents an approach to deriving a learner model directly from an assessment rubric.
We illustrate how the approach can be applied to automatize the human assessment of an activity developed for testing computational thinking skills.
arXiv Detail & Related papers (2022-09-07T10:09:12Z) - Application of Deep Self-Attention in Knowledge Tracing [2.5852720579998336]
This paper proposed Deep Self-Attentive Knowledge Tracing (DSAKT) based on the data of PTA, an online assessment system used by students in many universities in China.
Experimentation on the data of PTA shows that DSAKT outperforms the other models for knowledge tracing an improvement of 2.1% on average.
arXiv Detail & Related papers (2021-05-17T14:45:38Z) - Student sentiment Analysis Using Classification With Feature Extraction
Techniques [0.0]
This paper describes the web-based learning and their effectiveness towards students.
We worked on how machine learning techniques like Logistic Regression (LR), Support Vector Machine (SVM), Naive Bayes (NB), Decision Tree (DT)
arXiv Detail & Related papers (2021-02-01T18:48:06Z) - Automated Personalized Feedback Improves Learning Gains in an
Intelligent Tutoring System [34.19909376464836]
We investigate how automated, data-driven, personalized feedback in a large-scale intelligent tutoring system (ITS) improves student learning outcomes.
We propose a machine learning approach to generate personalized feedback, which takes individual needs of students into account.
We utilize state-of-the-art machine learning and natural language processing techniques to provide the students with personalized hints, Wikipedia-based explanations, and mathematical hints.
arXiv Detail & Related papers (2020-05-05T18:30:08Z) - Knowledge-guided Deep Reinforcement Learning for Interactive
Recommendation [49.32287384774351]
Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy.
We propose Knowledge-Guided deep Reinforcement learning to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation.
arXiv Detail & Related papers (2020-04-17T05:26:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.