Human-Centered Design for AI-based Automatically Generated Assessment Reports: A Systematic Review

• URL: http://arxiv.org/abs/2501.00081v1
• Date: Mon, 30 Dec 2024 16:20:07 GMT
• Title: Human-Centered Design for AI-based Automatically Generated Assessment Reports: A Systematic Review
• Authors: Ehsan Latif, Ying Chen, Xiaoming Zhai, Yue Yin,
• Abstract summary: This study emphasizes the importance of reducing teachers' cognitive demands through user-centered and intuitive designs. It highlights the potential of diverse information presentation formats such as text, visual aids, and plots and advanced functionalities such as live and interactive features to enhance usability. The framework aims to address challenges in engaging teachers with technology-enhanced assessment results, facilitating data-driven decision-making, and providing personalized feedback to improve the teaching and learning process.
• Score: 4.974197456441281
• License:
• Abstract: This paper provides a comprehensive review of the design and implementation of automatically generated assessment reports (AutoRs) for formative use in K-12 Science, Technology, Engineering, and Mathematics (STEM) classrooms. With the increasing adoption of technology-enhanced assessments, there is a critical need for human-computer interactive tools that efficiently support the interpretation and application of assessment data by teachers. AutoRs are designed to provide synthesized, interpretable, and actionable insights into students' performance, learning progress, and areas for improvement. Guided by cognitive load theory, this study emphasizes the importance of reducing teachers' cognitive demands through user-centered and intuitive designs. It highlights the potential of diverse information presentation formats such as text, visual aids, and plots and advanced functionalities such as live and interactive features to enhance usability. However, the findings also reveal that many existing AutoRs fail to fully utilize these approaches, leading to high initial cognitive demands and limited engagement. This paper proposes a conceptual framework to inform the design, implementation, and evaluation of AutoRs, balancing the trade-offs between usability and functionality. The framework aims to address challenges in engaging teachers with technology-enhanced assessment results, facilitating data-driven decision-making, and providing personalized feedback to improve the teaching and learning process.

Related papers

• Real-time classification of EEG signals using Machine Learning deployment [0.0]
  This study proposes a machine learning-based approach for predicting the level of students' comprehension with regard to a certain topic. A browser interface was introduced that accesses the values of the system's parameters to determine a student's level of concentration on a chosen topic.
  arXiv  Detail & Related papers  (2024-12-27T08:14:28Z)
• GUI Agents: A Survey [129.94551809688377]
  Graphical User Interface (GUI) agents, powered by Large Foundation Models, have emerged as a transformative approach to automating human-computer interaction. Motivated by the growing interest and fundamental importance of GUI agents, we provide a comprehensive survey that categorizes their benchmarks, evaluation metrics, architectures, and training methods.
  arXiv  Detail & Related papers  (2024-12-18T04:48:28Z)
• An Automated Explainable Educational Assessment System Built on LLMs [12.970776782360366]
  AERA Chat is an automated educational assessment system designed for interactive and visual evaluations of student responses. Our system allows users to input questions and student answers, providing educators and researchers with insights into assessment accuracy.
  arXiv  Detail & Related papers  (2024-12-17T23:29:18Z)
• AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment [12.970776782360366]
  AERA Chat is an interactive platform to provide visually explained assessment of student answers. Users can input questions and student answers to obtain automated, explainable assessment results from large language models.
  arXiv  Detail & Related papers  (2024-10-12T11:57:53Z)
• Lessons Learned from Designing an Open-Source Automated Feedback System for STEM Education [5.326069675013602]
  We present RATsApp, an open-source automated feedback system (AFS) that incorporates research-based features such as formative feedback. The system focuses on core STEM competencies such as mathematical competence, representational competence, and data literacy. As an open-source platform, RATsApp encourages public contributions to its ongoing development, fostering a collaborative approach to improve educational tools.
  arXiv  Detail & Related papers  (2024-01-19T07:13:07Z)
• K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning [83.19215082550163]
  We propose K-ESConv, a novel prompt learning based knowledge injection method for emotional support dialogue system. We evaluate our model on an emotional support dataset ESConv, where the model retrieves and incorporates knowledge from external professional emotional Q&A forum.
  arXiv  Detail & Related papers  (2023-12-16T08:10:10Z)
• Towards Goal-oriented Intelligent Tutoring Systems in Online Education [69.06930979754627]
  We propose a new task, named Goal-oriented Intelligent Tutoring Systems (GITS) GITS aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment. We propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI)
  arXiv  Detail & Related papers  (2023-12-03T12:37:16Z)
• Empowering Private Tutoring by Chaining Large Language Models [87.76985829144834]
  This work explores the development of a full-fledged intelligent tutoring system powered by state-of-the-art large language models (LLMs) The system is into three inter-connected core processes-interaction, reflection, and reaction. Each process is implemented by chaining LLM-powered tools along with dynamically updated memory modules.
  arXiv  Detail & Related papers  (2023-09-15T02:42:03Z)
• PapagAI:Automated Feedback for Reflective Essays [48.4434976446053]
  We present the first open-source automated feedback tool based on didactic theory and implemented as a hybrid AI system. The main objective of our work is to enable better learning outcomes for students and to complement the teaching activities of lecturers.
  arXiv  Detail & Related papers  (2023-07-10T11:05:51Z)
• Learning Knowledge Representation with Meta Knowledge Distillation for Single Image Super-Resolution [82.89021683451432]
  We propose a model-agnostic meta knowledge distillation method under the teacher-student architecture for the single image super-resolution task. Experiments conducted on various single image super-resolution datasets demonstrate that our proposed method outperforms existing defined knowledge representation related distillation methods.
  arXiv  Detail & Related papers  (2022-07-18T02:41:04Z)
• Demonstrating REACT: a Real-time Educational AI-powered Classroom Tool [0.9899017174990579]
  We present a new Real-time Educational AI-powered Classroom Tool that employs EDM techniques for supporting the decision-making process of educators. ReACT is a data-driven tool with a user-friendly graphical interface. It analyzes students' performance data and provides context-based alerts as well as recommendations to educators for course planning.
  arXiv  Detail & Related papers  (2021-07-30T03:09:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.