Impact of combining human and analytics feedback on students' engagement
with, and performance in, reflective writing tasks
- URL: http://arxiv.org/abs/2211.08222v1
- Date: Tue, 15 Nov 2022 15:41:26 GMT
- Title: Impact of combining human and analytics feedback on students' engagement
with, and performance in, reflective writing tasks
- Authors: Wannapon Suraworachet, Qi Zhou and Mutlu Cukurova
- Abstract summary: This study proposes a personalised behavioural feedback intervention based on students' writing engagement analytics.
In a semester-long experimental study involving 81 postgraduate students, its impact on learning engagement and performance was studied.
- Score: 3.4843936798388015
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Reflective writing is part of many higher education courses across the globe.
It is often considered a challenging task for students as it requires
self-regulated learning skills to appropriately plan, timely engage and deeply
reflect on learning experiences. Despite an advance in writing analytics and
the pervasiveness of human feedback aimed to support student reflections,
little is known about how to integrate feedback from humans and analytics to
improve students' learning engagement and performance in reflective writing
tasks. This study proposes a personalised behavioural feedback intervention
based on students' writing engagement analytics utilising time-series analysis
of digital traces from a ubiquitous online word processing platform. In a
semester-long experimental study involving 81 postgraduate students, its impact
on learning engagement and performance was studied. The results showed that the
intervention cohort engaged statistically significantly more in their
reflective writing task after receiving the combined feedback compared to the
control cohort which only received human feedback on their reflective writing
content. Further analyses revealed that the intervention cohort reflected more
regularly at the weekly level, the regularity of weekly reflection led to
better performance grades, and the impact on students with low self-regulated
learning skills was higher. This study emphasizes the powerful benefits of
implementing combined feedback approaches in which the strengths of analytics
and human feedback are synthesized to improve student engagement and
performance. Further research should explore the long-term sustainability of
the observed effects and their validity in other contexts.
Related papers
- Evaluating Pedagogical Incentives in Undergraduate Computing: A Mixed Methods Approach Using Learning Analytics [0.0]
This paper assesses the impact of new pedagogical incentives implemented in a first-year undergraduate computing module at University College London.
We employ a mixed methods approach, combining learning analytics with qualitative data to evaluate the effectiveness of these incentives on increasing student engagement.
Our paper introduces an interpretable and actionable model for student engagement, which integrates objective, data-driven analysis with students' perspectives.
arXiv Detail & Related papers (2024-03-13T16:39:38Z) - Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes.
We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function.
Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
arXiv Detail & Related papers (2024-03-05T09:09:15Z) - Scalable Two-Minute Feedback: Digital, Lecture-Accompanying Survey as a Continuous Feedback Instrument [0.0]
Detailed feedback on courses and lecture content is essential for their improvement and also serves as a tool for reflection.
The article used a digital survey format as formative feedback which attempts to measure student stress in a quantitative part and to address the participants' reflection in a qualitative part.
The results show a low, but constant rate of feedback. Responses mostly cover topics of the lecture content or organizational aspects and were intensively used to report issues within the lecture.
arXiv Detail & Related papers (2023-10-30T08:14:26Z) - Sensitivity, Performance, Robustness: Deconstructing the Effect of
Sociodemographic Prompting [64.80538055623842]
sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give.
We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
arXiv Detail & Related papers (2023-09-13T15:42:06Z) - Improving Factuality of Abstractive Summarization via Contrastive Reward
Learning [77.07192378869776]
We propose a simple but effective contrastive learning framework that incorporates recent developments in reward learning and factuality metrics.
Empirical studies demonstrate that the proposed framework enables summarization models to learn from feedback of factuality metrics.
arXiv Detail & Related papers (2023-07-10T12:01:18Z) - Factually Consistent Summarization via Reinforcement Learning with
Textual Entailment Feedback [57.816210168909286]
We leverage recent progress on textual entailment models to address this problem for abstractive summarization systems.
We use reinforcement learning with reference-free, textual entailment rewards to optimize for factual consistency.
Our results, according to both automatic metrics and human evaluation, show that our method considerably improves the faithfulness, salience, and conciseness of the generated summaries.
arXiv Detail & Related papers (2023-05-31T21:04:04Z) - An Interdisciplinary Perspective on Evaluation and Experimental Design
for Visual Text Analytics: Position Paper [24.586485898038312]
In this paper, we focus on the issues of evaluating visual text analytics approaches.
We identify four key groups of challenges for evaluating visual text analytics approaches.
arXiv Detail & Related papers (2022-09-23T11:47:37Z) - A literature survey on student feedback assessment tools and their usage
in sentiment analysis [0.0]
We evaluate the effectiveness of various in-class feedback assessment methods such as Kahoot!, Mentimeter, Padlet, and polling.
We propose a sentiment analysis model for extracting the explicit suggestions from the students' qualitative feedback comments.
arXiv Detail & Related papers (2021-09-09T06:56:30Z) - The Challenges of Assessing and Evaluating the Students at Distance [77.34726150561087]
The COVID-19 pandemic has caused a strong effect on higher education institutions with the closure of classroom teaching activities.
This short essay aims to explore the challenges posed to Portuguese higher education institutions and to analyze the challenges posed to evaluation models.
arXiv Detail & Related papers (2021-01-30T13:13:45Z) - Facial Feedback for Reinforcement Learning: A Case Study and Offline
Analysis Using the TAMER Framework [51.237191651923666]
We investigate the potential of agent learning from trainers' facial expressions via interpreting them as evaluative feedback.
With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback.
Our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible.
arXiv Detail & Related papers (2020-01-23T17:50:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.