Assessment in Team Problem-Solving Exercises in Computing Education

• URL: http://arxiv.org/abs/2607.19209v1
• Date: Tue, 21 Jul 2026 15:40:16 GMT
• Title: Assessment in Team Problem-Solving Exercises in Computing Education
• Abstract summary: This paper presents methods for assessing student teams in tabletop exercises (TTXs)<n>TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents.<n>We compare two post-TTX team assessment methods -- clustering and large language models (LLMs)<n>We evaluate these methods against instructor-assigned scores based on standardized rubrics.
• Score: 0.3283937504286783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents. While assessment is essential for determining how well teams achieve learning objectives, the complex, open-ended nature of TTXs often leads to delayed or incomplete feedback. TTX learning platforms can record teams' actions and communication; yet, leveraging these data to assess performance is underexplored. To address this gap, we compared two post-TTX team assessment methods -- clustering and large language models (LLMs) -- using an original dataset from 81 participants across two countries. We evaluated these methods against instructor-assigned scores based on standardized rubrics. Clustering grouped teams that approached TTX tasks similarly, enabling instructors to deliver faster, targeted feedback to teams within a cluster. This method was valid and reliable, with low computational requirements. LLMs used the standardized rubrics to assess teams' communication. While GPT-4o frequently disagreed with instructor scores, GPT-5.2 demonstrated considerably lower error. The researched methods have been integrated into INJECT, an open-source TTX learning platform, to support scalability and teaching practice. To encourage community adoption, we publicly share all datasets, software tools, and a full-fledged TTX scenario.

Related papers

• Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned [0.13999481573773068]
  This paper examines the integration of technology-enhanced tabletop exercises (TTXs) into computing education, focusing on cybersecurity curricula.<n>Although TTXs are well-established in professional practice, they remain underused in universities.<n>We address this gap by augmenting TTX delivery and evaluation through the INJECT Exercise Platform (IXP), a web-based environment that automates scenario flow and enables data-driven assessment.
  arXiv  Detail & Related papers  (2026-07-30T13:17:12Z)
• From Paper to Platform: Evolution of a Novel Learning Environment for Tabletop Exercises [0.2796197251957245]
  This paper presents data and teaching experience from a cybersecurity course that introduces tabletop exercises in classrooms. InJECT Exercise Platform (IXP) is a web-based learning environment for delivering and evaluating the exercises. Unlike in traditional tabletop exercises, which are difficult to evaluate manually, IXP provides insights into students' behavior and learning.
  arXiv  Detail & Related papers  (2024-04-17T01:52:48Z)
• Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching [0.0]
  Student evaluations of teaching (SETs) are important sources of feedback for educators. A collection of SETs can also be useful to administrators as signals for courses or entire programs. We discuss a novel method for analyzing SETs using natural language processing (NLP) and large language models (LLMs)
  arXiv  Detail & Related papers  (2024-03-18T17:21:35Z)
• Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning [85.66907881270785]
  We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training. In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk. In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
  arXiv  Detail & Related papers  (2024-01-07T18:12:20Z)
• ClusT3: Information Invariant Test-Time Training [19.461441044484427]
  Test-time training (TTT) methods have been developed in an attempt to mitigate these vulnerabilities. We propose a novel unsupervised TTT technique based on the Mutual of Mutual Information between multi-scale feature maps and a discrete latent representation. Experimental results demonstrate competitive classification performance on different popular test-time adaptation benchmarks.
  arXiv  Detail & Related papers  (2023-10-18T21:43:37Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of supervised fine-tuning (SFT)<n>We also review the potential pitfalls of SFT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)
• Evaluating Language Models for Mathematics through Interactions [116.67206980096513]
  We introduce CheckMate, a prototype platform for humans to interact with and evaluate large language models (LLMs) We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics. We derive a taxonomy of human behaviours and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness.
  arXiv  Detail & Related papers  (2023-06-02T17:12:25Z)
• Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training [37.75537703971045]
  We develop a test-time anchored clustering (TTAC) approach to enable stronger test-time feature learning. Self-training(ST) has demonstrated great success in learning from unlabeled data. TTAC++ consistently outperforms the state-of-the-art methods on five TTT datasets.
  arXiv  Detail & Related papers  (2023-03-20T04:30:18Z)
• How Does In-Context Learning Help Prompt Tuning? [55.78535874154915]
  Fine-tuning large language models is becoming ever more impractical due to their rapidly-growing scale. This motivates the use of parameter-efficient adaptation methods such as prompt tuning (PT), which adds a small number of tunable embeddings to an otherwise frozen model. Recently, Singhal et al. (2022) propose instruction prompt tuning'' (IPT), which combines PT with ICL by concatenating a natural language demonstration with learned prompt embeddings.
  arXiv  Detail & Related papers  (2023-02-22T17:45:12Z)
• TeST: Test-time Self-Training under Distribution Shift [99.68465267994783]
  Test-Time Self-Training (TeST) is a technique that takes as input a model trained on some source data and a novel data distribution at test time. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms.
  arXiv  Detail & Related papers  (2022-09-23T07:47:33Z)
• SETSum: Summarization and Visualization of Student Evaluations of Teaching [74.76373136325032]
  Student Evaluations of Teaching (SETs) are widely used in colleges and universities. SETSum provides organized illustrations of SET findings to instructors and other reviewers.
  arXiv  Detail & Related papers  (2022-07-08T01:40:11Z)
• Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering [37.76664203157892]
  We develop a test-time anchored clustering (TTAC) approach to enable stronger test-time feature learning. TTAC discovers clusters in both source and target domain and match the target clusters to the source ones to improve generalization. We demonstrate that under all TTT protocols TTAC consistently outperforms the state-of-the-art methods on five TTT datasets.
  arXiv  Detail & Related papers  (2022-06-06T16:23:05Z)
• Secure Distributed Training at Scale [65.7538150168154]
  Training in presence of peers requires specialized distributed training algorithms with Byzantine tolerance. We propose a novel protocol for secure (Byzantine-tolerant) decentralized training that emphasizes communication efficiency.
  arXiv  Detail & Related papers  (2021-06-21T17:00:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.