Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistant

• URL: http://arxiv.org/abs/2307.04276v1
• Date: Sun, 9 Jul 2023 23:02:19 GMT
• Title: Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistant
• Authors: Yann Hicke, Tonghua Tian, Karan Jha, Choong Hee Kim
• Abstract summary: We present a transformer-based architecture capable of achieving above-human accuracy in annotating argumentative writing discourse elements. We expand on planned future work investigating the explainability of our model so that actionable feedback can be offered to the student.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated Essay scoring has been explored as a research and industry problem for over 50 years. It has drawn a lot of attention from the NLP community because of its clear educational value as a research area that can engender the creation of valuable time-saving tools for educators around the world. Yet, these tools are generally focused on detecting good grammar, spelling mistakes, and organization quality but tend to fail at incorporating persuasiveness features in their final assessment. The responsibility to give actionable feedback to the student to improve the strength of their arguments is left solely on the teacher's shoulders. In this work, we present a transformer-based architecture capable of achieving above-human accuracy in annotating argumentative writing discourse elements for their persuasiveness quality and we expand on planned future work investigating the explainability of our model so that actionable feedback can be offered to the student and thus potentially enable a partnership between the teacher's advice and the machine's advice.

Related papers

• "My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays [6.810086342993699]
  This paper introduces CAELF, a Contestable AI Empowered LLM Framework for automating interactive feedback. CAELF allows students to query, challenge, and clarify their feedback by integrating a multi-agent system with computational argumentation. A case study on 500 critical thinking essays with user studies demonstrates that CAELF significantly improves interactive feedback.
  arXiv  Detail & Related papers  (2024-09-11T17:59:01Z)
• Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers [6.161370712594005]
  We have 742 student responses ranging over 75 courses in a Computer Science department. For each course, we synthesise a summary of the course evaluations and actionable items for the instructor. Our work highlights the possibility of using generative AI to produce factual, actionable, and appropriate feedback for teachers in the classroom setting.
  arXiv  Detail & Related papers  (2024-07-01T13:29:55Z)
• Representational Alignment Supports Effective Machine Teaching [81.19197059407121]
  We integrate insights from machine teaching and pragmatic communication with the literature on representational alignment. We design a supervised learning environment that disentangles representational alignment from teacher accuracy.
  arXiv  Detail & Related papers  (2024-06-06T17:48:24Z)
• Evaluation of ChatGPT Feedback on ELL Writers' Coherence and Cohesion [0.7028778922533686]
  ChatGPT has had a transformative effect on education where students are using it to help with homework assignments and teachers are actively employing it in their teaching practices. This study evaluated the quality of the feedback generated by ChatGPT regarding the coherence and cohesion of the essays written by English Language learners (ELLs) students.
  arXiv  Detail & Related papers  (2023-10-10T10:25:56Z)
• PapagAI:Automated Feedback for Reflective Essays [48.4434976446053]
  We present the first open-source automated feedback tool based on didactic theory and implemented as a hybrid AI system. The main objective of our work is to enable better learning outcomes for students and to complement the teaching activities of lecturers.
  arXiv  Detail & Related papers  (2023-07-10T11:05:51Z)
• Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment [75.59538732476346]
  We focus on the problem of generating such gap-focused questions (GFQs) automatically. We define the task, highlight key desired aspects of a good GFQ, and propose a model that satisfies these.
  arXiv  Detail & Related papers  (2023-07-06T22:21:42Z)
• Perception, performance, and detectability of conversational artificial intelligence across 32 university courses [15.642614735026106]
  We compare the performance of ChatGPT against students on 32 university-level courses. We find that ChatGPT's performance is comparable, if not superior, to that of students in many courses. We find an emerging consensus among students to use the tool, and among educators to treat this as plagiarism.
  arXiv  Detail & Related papers  (2023-05-07T10:37:51Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)
• Automated Evaluation for Student Argumentative Writing: A Survey [2.9466390764652415]
  This paper surveys and organizes research works in an under-studied area, which we call automated evaluation for student argumentative writing. Unlike traditional automated writing evaluation that focuses on holistic essay scoring, this field is more specific: it focuses on evaluating argumentative essays and offers specific feedback.
  arXiv  Detail & Related papers  (2022-05-09T07:27:59Z)
• My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism [71.34160809068996]
  Recent work shows that automated scoring systems are prone to even common-sense adversarial samples. We utilize recent advances in interpretability to find the extent to which features such as coherence, content and relevance are important for automated scoring mechanisms. We also find that since the models are not semantically grounded with world-knowledge and common sense, adding false facts such as the world is flat'' actually increases the score instead of decreasing it.
  arXiv  Detail & Related papers  (2020-12-27T06:19:20Z)
• Neural Multi-Task Learning for Teacher Question Detection in Online Classrooms [50.19997675066203]
  We build an end-to-end neural framework that automatically detects questions from teachers' audio recordings. By incorporating multi-task learning techniques, we are able to strengthen the understanding of semantic relations among different types of questions.
  arXiv  Detail & Related papers  (2020-05-16T02:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.