A survey on grading format of automated grading tools for programming
assignments
- URL: http://arxiv.org/abs/2212.01714v1
- Date: Sun, 4 Dec 2022 00:49:16 GMT
- Title: A survey on grading format of automated grading tools for programming
assignments
- Authors: Aditi Agrawal, Benjamin Reed
- Abstract summary: The prevalence of online platforms and studies has generated the demand for automated grading tools.
This survey studies and evaluates the automated grading tools based on their evaluation format.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The prevalence of online platforms and studies has generated the demand for
automated grading tools, and as a result, there are plenty in the market. Such
tools are developed to grade coding assignments quickly, accurately, and
effortlessly. Since there are varieties of tools available to cater to the
diverse options of programming languages and concepts, it is overwhelming for
any instructor to decide which one suits one's requirements. There are several
surveys studying the tools and giving insights into how they function and what
they support. However other than knowing the functionality, it is important for
an instructor to know how the assignments are graded and what is the format of
the test cases. This is crucial since the instructor has to design the grading
format and therefore requires a learning curve. This survey studies and
evaluates the automated grading tools based on their evaluation format. This in
turn helps a reader in deciding which tool to choose and provides an insight
into what are the assessment settings and approaches used in grading the coding
assignment in any specific grading tool.
Related papers
- Creating a Trajectory for Code Writing: Algorithmic Reasoning Tasks [0.923607423080658]
This paper describes instruments and the machine learning models used for validating them.
We have used the data collected in an introductory programming course in the penultimate week of the semester.
Preliminary research suggests ART type instruments can be combined with specific machine learning models to act as an effective learning trajectory.
arXiv Detail & Related papers (2024-04-03T05:07:01Z) - What Are Tools Anyway? A Survey from the Language Model Perspective [67.18843218893416]
Language models (LMs) are powerful yet mostly for text generation tasks.
We provide a unified definition of tools as external programs used by LMs.
We empirically study the efficiency of various tooling methods.
arXiv Detail & Related papers (2024-03-18T17:20:07Z) - TOOLVERIFIER: Generalization to New Tools via Self-Verification [69.85190990517184]
We introduce a self-verification method which distinguishes between close candidates by self-asking contrastive questions during tool selection.
Experiments on 4 tasks from the ToolBench benchmark, consisting of 17 unseen tools, demonstrate an average improvement of 22% over few-shot baselines.
arXiv Detail & Related papers (2024-02-21T22:41:38Z) - Automated Grading and Feedback Tools for Programming Education: A
Systematic Review [7.776434991976473]
Most papers assess the correctness of assignments in object-oriented languages.
Few tools assess the maintainability, readability or documentation of the source code.
Most tools offered fully automated assessment to allow for near-instantaneous feedback.
arXiv Detail & Related papers (2023-06-20T17:54:50Z) - ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented
Visual Models [102.63817106363597]
We build ELEVATER, the first benchmark to compare and evaluate pre-trained language-augmented visual models.
It consists of 20 image classification datasets and 35 object detection datasets, each of which is augmented with external knowledge.
We will release our toolkit and evaluation platforms for the research community.
arXiv Detail & Related papers (2022-04-19T10:23:42Z) - An Analysis of Programming Course Evaluations Before and After the
Introduction of an Autograder [1.329950749508442]
This paper studies the answers to the standardized university evaluation questionnaires of foundational computer science courses which recently introduced autograding.
We hypothesize how the autograder might have contributed to the significant changes in the data, such as, improved interactions between tutors and students, improved overall course quality, improved learning success, increased time spent, and reduced difficulty.
The autograder technology can be validated as a teaching method to improve student satisfaction with programming courses.
arXiv Detail & Related papers (2021-10-28T14:09:44Z) - Using Personality Detection Tools for Software Engineering Research: How
Far Can We Go? [12.56413718364189]
Self-assessment questionnaires are not a practical solution for collecting multiple observations on a large scale.
Off-the-shelf solutions trained on non-technical corpora might not be readily applicable to technical domains like Software Engineering.
arXiv Detail & Related papers (2021-10-11T07:02:34Z) - ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
In this paper, we frame the problem of providing feedback as few-shot classification.
A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors.
Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
arXiv Detail & Related papers (2021-07-23T22:41:28Z) - Get It Scored Using AutoSAS -- An Automated System for Scoring Short
Answers [63.835172924290326]
We present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS)
We propose and explain the design and development of a system for SAS, namely AutoSAS.
AutoSAS shows state-of-the-art performance and achieves better results by over 8% in some of the question prompts.
arXiv Detail & Related papers (2020-12-21T10:47:30Z) - Scaling Systematic Literature Reviews with Machine Learning Pipelines [57.82662094602138]
Systematic reviews entail the extraction of data from scientific documents.
We construct a pipeline that automates each of these aspects, and experiment with many human-time vs. system quality trade-offs.
We find that we can get surprising accuracy and generalisability of the whole pipeline system with only 2 weeks of human-expert annotation.
arXiv Detail & Related papers (2020-10-09T16:19:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.