Learning Critical Testing Literacy Through Puzzles: an Experience Report

• URL: http://arxiv.org/abs/2606.20129v1
• Date: Thu, 18 Jun 2026 11:53:00 GMT
• Title: Learning Critical Testing Literacy Through Puzzles: an Experience Report
• Abstract summary: In this paper, we report our experiences and takeaways from workshops using puzzles to learn CTL.<n>We conducted thirteen workshops with students, testers, teachers, and primary school pupils to assess puzzle-based teaching of critical testing literacy.
• Score: 0.5693122506667715
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we report our experiences and takeaways from workshops using puzzles to learn CTL. Background: Software testing is important yet difficult to teach. We introduced a BoK of puzzle-based learning activities to teach CTL, based on a model of critical tester's cognition, leading to the pedagogical framework P4TEST. We conducted thirteen workshops with students, testers, teachers, and primary school pupils to assess puzzle-based teaching of critical testing literacy. Experience: Across eleven workshops, we used a semi-structured approach, varying puzzles, materials, and timing. In two additional workshops, we introduced workbooks and think-aloud sessions to gather more data on the learning experience. Observations: Participants consistently perceived themselves as experimenting while solving puzzles. Students tended to converge on solutions, while professionals continued exploring. Emotions were visible in behaviour but hard to surface through written reflection alone. Think-aloud sessions revealed immediate reasoning; written reflections elicited more meta-cognitive reflection. The theme Sensemaking / reflection-in-action captured how participants framed problems, navigated dead ends, and shifted strategies. Reflections: Puzzles are not the intervention: the entire sequence of solving, debriefing, and reflecting is. Designing that sequence more deliberately is the work ahead. We also developed an open-source web application with built-in analytics to customise workshops.

Related papers

• Writing Blog Posts Helps Students Connect Experiential Learning to the Workplace [2.695886599085046]
  The study included twenty-five juniors and seniors studying CS at CTCs and other affordable local colleges.<n>Each student was assigned a simple open source issue to solve from a popular open source project over the course of several weeks with the help of an industry mentor.<n>While working on the project, students drafted a LinkedIn blog post using a five-section outline covering project mission, assigned issue, technical architecture, challenges faced, and submitted solution.<n>Four themes emerged from the posts: identifying problem solving techniques, growth mindset, the challenges and benefits of collaborative development, and the impacts of their contribution on users.
  arXiv  Detail & Related papers  (2026-04-21T00:05:19Z)
• Moving Beyond Review: Applying Language Models to Planning and Translation in Reflection [17.23739251695297]
  We propose the first application of large language models (LLMs) to the planning and translation steps of reflective writing.<n>Pensée is a tool to explore the effects of explicit AI support during these stages by scaffolding structured reflection planning using a conversational agent.<n>Results show significantly greater reflection depth and structural quality when learners receive support during planning and translation stages of Cognitive Process Theory of writing.
  arXiv  Detail & Related papers  (2026-03-30T15:42:38Z)
• Understanding Student Interaction with AI-Powered Next-Step Hints: Strategies and Challenges [1.446446435461625]
  Next-step hint feedback provides students with actionable steps to progress towards solving programming tasks.<n>This study investigates how students interact with an AI-driven next-step hint system in an in-IDE learning environment.
  arXiv  Detail & Related papers  (2025-11-09T12:56:34Z)
• Student Engagement with GenAI's Tutoring Feedback: A Mixed Methods Study [0.0]
  The research aims to: (1) identify what students think when they engage with the tutoring feedback components, and (2) explore the relations between the feedback components, students' visual attention, verbalized thoughts, and their immediate actions as part of the problem-solving process.<n>The analysis of students' thoughts while engaging with 380 feedback components revealed four main themes: students express understanding or disagreement, additional information needed, and students explicitly judge the feedback.
  arXiv  Detail & Related papers  (2025-09-26T22:17:20Z)
• Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles [40.63842379016203]
  This paper provides a comprehensive study of rule-based visual RL, using jigsaw puzzles as a structured experimental framework.<n> Jigsaw puzzles offer inherent ground truth, adjustable difficulty, and demand complex decision-making.<n> MLLMs, initially performing near to random guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and generalize to complex, unseen configurations through fine-tuning.
  arXiv  Detail & Related papers  (2025-05-29T16:01:22Z)
• Playpen: An Environment for Exploring Learning Through Conversational Interaction [84.0413820245725]
  We investigate whether Dialogue Games can also serve as a source of feedback signals for learning.<n>We introduce Playpen, an environment for off- and online learning through Dialogue Game self-play.<n>We find that imitation learning through SFT improves performance on unseen instances, but negatively impacts other skills.
  arXiv  Detail & Related papers  (2025-04-11T14:49:33Z)
• BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts [59.83547898874152]
  BloomWise is a cognitively-inspired prompting technique for large language models (LLMs)<n>It is designed to enhance LLMs' performance on mathematical problem solving while making their solutions more explainable.
  arXiv  Detail & Related papers  (2024-10-05T09:27:52Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• YODA: Teacher-Student Progressive Learning for Language Models [82.0172215948963]
  This paper introduces YODA, a teacher-student progressive learning framework. It emulates the teacher-student education process to improve the efficacy of model fine-tuning. Experiments show that training LLaMA2 with data from YODA improves SFT with significant performance gain.
  arXiv  Detail & Related papers  (2024-01-28T14:32:15Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)
• Teachable Reinforcement Learning via Advice Distillation [161.43457947665073]
  We propose a new supervision paradigm for interactive learning based on "teachable" decision-making systems that learn from structured advice provided by an external teacher. We show that agents that learn from advice can acquire new skills with significantly less human supervision than standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-03-19T03:22:57Z)
• Interaction-limited Inverse Reinforcement Learning [50.201765937436654]
  We present two different training strategies: Curriculum Inverse Reinforcement Learning (CIRL) covering the teacher's perspective, and Self-Paced Inverse Reinforcement Learning (SPIRL) focusing on the learner's perspective. Using experiments in simulations and experiments with a real robot learning a task from a human demonstrator, we show that our training strategies can allow a faster training than a random teacher for CIRL and than a batch learner for SPIRL.
  arXiv  Detail & Related papers  (2020-07-01T12:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.