Towards Generalizable Detection of Urgency of Discussion Forum Posts
- URL: http://arxiv.org/abs/2307.07614v1
- Date: Fri, 14 Jul 2023 20:21:50 GMT
- Title: Towards Generalizable Detection of Urgency of Discussion Forum Posts
- Authors: Valdemar \v{S}v\'abensk\'y, Ryan S. Baker, Andr\'es Zambrano, Yishan
Zou, Stefan Slater
- Abstract summary: Students who take an online course, such as a MOOC, use the course's discussion forum to ask questions or reach out to instructors when encountering an issue.
We build predictive models that automatically determine the urgency of each forum post, so that these posts can be brought to instructors' attention.
This paper goes beyond previous work by predicting not just a binary decision cut-off but a post's level of urgency on a 7-point scale.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Students who take an online course, such as a MOOC, use the course's
discussion forum to ask questions or reach out to instructors when encountering
an issue. However, reading and responding to students' questions is difficult
to scale because of the time needed to consider each message. As a result,
critical issues may be left unresolved, and students may lose the motivation to
continue in the course. To help address this problem, we build predictive
models that automatically determine the urgency of each forum post, so that
these posts can be brought to instructors' attention. This paper goes beyond
previous work by predicting not just a binary decision cut-off but a post's
level of urgency on a 7-point scale. First, we train and cross-validate several
models on an original data set of 3,503 posts from MOOCs at University of
Pennsylvania. Second, to determine the generalizability of our models, we test
their performance on a separate, previously published data set of 29,604 posts
from MOOCs at Stanford University. While the previous work on post urgency used
only one data set, we evaluated the prediction across different data sets and
courses. The best-performing model was a support vector regressor trained on
the Universal Sentence Encoder embeddings of the posts, achieving an RMSE of
1.1 on the training set and 1.4 on the test set. Understanding the urgency of
forum posts enables instructors to focus their time more effectively and, as a
result, better support student learning.
Related papers
- SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights [89.56181323849512]
We propose SuperCorrect, a framework that uses a large teacher model to supervise and correct both the reasoning and reflection processes of a smaller student model.
In the first stage, we extract hierarchical high-level and detailed thought templates from the teacher model to guide the student model in eliciting more fine-grained reasoning thoughts.
In the second stage, we introduce cross-model collaborative direct preference optimization (DPO) to enhance the self-correction abilities of the student model.
arXiv Detail & Related papers (2024-10-11T17:25:52Z) - Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
Large language models (LLMs) present an opportunity to scale high-quality personalized education to all.
LLMs struggle to precisely detect student's errors and tailor their feedback to these errors.
Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
arXiv Detail & Related papers (2024-07-12T10:11:40Z) - ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
In this paper, we frame the problem of providing feedback as few-shot classification.
A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors.
Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
arXiv Detail & Related papers (2021-07-23T22:41:28Z) - Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need
in MOOC Forums [58.221459787471254]
Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility.
Due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support.
With the large volume of posts and high workloads for MOOC instructors, it is unlikely that the instructors can identify all learners requiring intervention.
This paper explores for the first time Bayesian deep learning on learner-based text posts with two methods: Monte Carlo Dropout and Variational Inference.
arXiv Detail & Related papers (2021-04-26T15:12:13Z) - Predicting student performance using data from an auto-grading system [0.0]
We build decision-tree and linear-regression models with various features extracted from the Marmoset auto-grading system.
We show that the linear-regression model using submission time intervals performs the best among all models in terms of Precision and F-Measure.
We also show that for students who are misclassified into poor-performance students, they have the lowest actual grades in the linear-regression model among all models.
arXiv Detail & Related papers (2021-02-02T03:02:39Z) - Learning Student Interest Trajectory for MOOCThread Recommendation [9.560312630927601]
We propose to predict future interest trajectories of students in Massive Open Online Courses (MOOCs)
Our model consists of two key operations: 1) Update operation and 2) Projection operation.
Our model significantly outperforms other baselines for thread recommendation.
arXiv Detail & Related papers (2021-01-10T10:23:11Z) - Unification of HDP and LDA Models for Optimal Topic Clustering of
Subject Specific Question Banks [55.41644538483948]
An increase in the popularity of online courses would result in an increase in the number of course-related queries for academics.
In order to reduce the time spent on answering each individual question, clustering them is an ideal choice.
We use the Hierarchical Dirichlet Process to determine an optimal topic number input for our LDA model runs.
arXiv Detail & Related papers (2020-10-04T18:21:20Z) - TACRED Revisited: A Thorough Evaluation of the TACRED Relation
Extraction Task [80.38130122127882]
TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE)
In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement?
We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.
arXiv Detail & Related papers (2020-04-30T15:07:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.