AI-Generated Interactive Fiction for Educational Use: A Pilot Study of Perceived Comprehensibility, Coherence, and Engagement

• URL: http://arxiv.org/abs/2608.10818v1
• Date: Tue, 11 Aug 2026 11:41:05 GMT
• Title: AI-Generated Interactive Fiction for Educational Use: A Pilot Study of Perceived Comprehensibility, Coherence, and Engagement
• Abstract summary: This paper presents a pilot user-centred evaluation of AI-generated interactive fiction (IF) for educational use in higher education.<n>We generated a controlled pool of scenarios and asked participants to play one generated episode and rate it on narrative clarity, story-content coherence, engagement, and length acceptance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative artificial intelligence (AI) can produce educational content at scale, including interactive and narrative learning experiences, but technical generation alone is not sufficient: scenarios that are confusing, narratively inconsistent, or unengaging are unlikely to be useful in practice. This paper presents a pilot user-centred evaluation of AI-generated interactive fiction (IF) for educational use in higher education. Using a previously described domain-agnostic pipeline and a shared STEM content base, we generated a controlled pool of scenarios and asked participants (N = 22, STEM higher-education) to play one generated episode and rate it on narrative clarity, story-content coherence, engagement, and length acceptance. A free-text prompt captured open feedback. Narrative clarity and length acceptance were rated positively, engagement sat near the neutral mid-point of the scale, and story-content coherence was the weakest dimension by a clear margin. Qualitative feedback points to quiz integration as the bottleneck. Artificial in-fiction motivation for quiz prompts and abrupt setting changes were reported. Feedback also pointed to missing story-level consequences for wrong answers. From these observations, we derive concrete design implications that can inform larger follow-up studies, including later work on learning effectiveness.

Related papers

• Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing [66.03354630509942]
  This paper reports on a study where we compare three designs for user-AI interaction in a learning context.<n>Fifty study participants were tasked with learning about nuclear safety protocols.
  arXiv  Detail & Related papers  (2026-09-01T02:24:35Z)
• PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation [8.432742005693283]
  We propose a knowledge textbfinformation-enhanced knowledge textbfdistillation method for textbfempathetic dialogue generation (PRIDE)<n>Our method leverages privileged information, such as expert psychological annotations or future event summaries, which is available exclusively during training but unavailable at inference time.<n>PRIDE has three key components: (1) An empathy-reasoning prompt that guides the teacher to explicitly decompose the empathetic process into understanding feelings and analyzing situations step-by-step; (2) A multi-source attention mechanism that directs the student to effectively integrate privileged information
  arXiv  Detail & Related papers  (2026-06-22T10:09:52Z)
• Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novels [50.43968216132018]
  We present an end-to-end system that transforms any literary work into an immersive, multi-character conversational experience.<n>This system is designed to solve two fundamental challenges for LLM-driven characters.
  arXiv  Detail & Related papers  (2025-12-08T11:57:46Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• WhatELSE: Shaping Narrative Spaces at Configurable Level of Abstraction for AI-bridged Interactive Storytelling [11.210282687859534]
  WhatELSE is an AI-bridged IN authoring system that creates narrative possibility spaces from example stories.<n>We show that WhatELSE enables authors to perceive and edit the narrative space and generates engaging interactive narratives at play-time.
  arXiv  Detail & Related papers  (2025-02-25T21:02:15Z)
• Data Science Principles for Interpretable and Explainable AI [0.7581664835990121]
  Interpretable and interactive machine learning aims to make complex models more transparent and controllable. This review synthesizes key principles from the growing literature in this field.
  arXiv  Detail & Related papers  (2024-05-17T05:32:27Z)
• Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems [23.093767743306973]
  Conversational tutoring systems (CTSs) offer learning experiences driven by natural language interaction. We introduce a novel type of CTS that leverages the recent advances in large language models (LLMs) in two ways. Ruffle&Riley allows a free-form conversation that follows the ITS-typical inner and outer loop structure.
  arXiv  Detail & Related papers  (2023-09-26T23:27:06Z)
• Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback [57.816210168909286]
  We leverage recent progress on textual entailment models to address this problem for abstractive summarization systems. We use reinforcement learning with reference-free, textual entailment rewards to optimize for factual consistency. Our results, according to both automatic metrics and human evaluation, show that our method considerably improves the faithfulness, salience, and conciseness of the generated summaries.
  arXiv  Detail & Related papers  (2023-05-31T21:04:04Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)
• DeltaScore: Fine-Grained Story Evaluation with Perturbations [69.33536214124878]
  We introduce DELTASCORE, a novel methodology that employs perturbation techniques for the evaluation of nuanced story aspects. Our central proposition posits that the extent to which a story excels in a specific aspect (e.g., fluency) correlates with the magnitude of its susceptibility to particular perturbations. We measure the quality of an aspect by calculating the likelihood difference between pre- and post-perturbation states using pre-trained language models.
  arXiv  Detail & Related papers  (2023-03-15T23:45:54Z)
• Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
  We present the first application of the BERTSum model to conversational language. We generate abstractive summaries of narrated instructional videos across a wide variety of topics. We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
  arXiv  Detail & Related papers  (2020-08-21T20:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.