Penny: Transition Network Analysis of Learner-Chatbot Interactions in Scaffolded EFL Writing

• URL: http://arxiv.org/abs/2607.14575v1
• Date: Thu, 16 Jul 2026 05:12:42 GMT
• Title: Penny: Transition Network Analysis of Learner-Chatbot Interactions in Scaffolded EFL Writing
• Abstract summary: This study uses Transition Network Analysis to model the temporal dynamics of Japanese EFL learners using "Penny"<n>Analysis of over 4,500 writing sessions and 21,000 interactions reveals two dominant behavioural loops.<n>Findings demonstrate that AI-scaffolded writing is a non-linear, dialogic process.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative AI chatbots promise to transform English as a Foreign Language (EFL) writing by providing immediate, personalised feedback. However, their pedagogical value depends on how learners engage with them - a process often treated as a "black box." This study uses Transition Network Analysis to model the temporal dynamics of Japanese EFL learners using "Penny," an LLM-powered writing chatbot. Analysis of over 4,500 writing sessions and 21,000 chatbot interactions reveals two dominant behavioural loops: a "Revision Loop," where feedback leads directly to successful error correction, and a "Chat Loop," where learners engage in sustained dialogue with the chatbot following feedback. Crucially, EFL proficiency significantly shapes interaction: high-proficiency learners engage more in open dialogue and negotiation with the chatbot, while low-proficiency learners rely more heavily on repetitive corrective feedback cycles. The findings demonstrate that AI-scaffolded writing is a non-linear, dialogic process and highlight the need for differentiated chatbot design to move beyond simple error correction and foster deeper cognitive engagement for all learners.

Related papers

• PythonPal: Enhancing Online Programming Education through Chatbot-Driven Personalized Feedback [0.0]
  PythonPal's design, featuring modules for conversation, tutorials, and exercises, was evaluated through student interactions and feedback.<n>Key findings reveal PythonPal's proficiency in syntax error recognition and user query comprehension.<n>Student feedback indicated satisfactory query understanding and feedback accuracy but also pointed out the need for faster responses and improved interaction quality.
  arXiv  Detail & Related papers  (2025-03-09T07:28:42Z)
• REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation [51.97224538045096]
  We introduce REALTALK, a 21-day corpus of authentic messaging app dialogues.<n>We compare EI attributes and persona consistency to understand the challenges posed by real-world dialogues.<n>Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation.
  arXiv  Detail & Related papers  (2025-02-18T20:29:01Z)
• "My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays [6.810086342993699]
  This paper introduces CAELF, a Contestable AI Empowered LLM Framework for automating interactive feedback. CAELF allows students to query, challenge, and clarify their feedback by integrating a multi-agent system with computational argumentation. A case study on 500 critical thinking essays with user studies demonstrates that CAELF significantly improves interactive feedback.
  arXiv  Detail & Related papers  (2024-09-11T17:59:01Z)
• Measuring and Controlling Instruction (In)Stability in Language Model Dialogs [72.38330196290119]
  System-prompting is a tool for customizing language-model chatbots, enabling them to follow a specific instruction. We propose a benchmark to test the assumption, evaluating instruction stability via self-chats. We reveal a significant instruction drift within eight rounds of conversations. We propose a lightweight method called split-softmax, which compares favorably against two strong baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:10:29Z)
• Multi-Purpose NLP Chatbot : Design, Methodology & Conclusion [0.0]
  This research paper provides a thorough analysis of the chatbots technology environment as it exists today. It provides a very flexible system that makes use of reinforcement learning strategies to improve user interactions and conversational experiences. The complexity of chatbots technology development is also explored in this study, along with the causes that have propelled these developments and their far-reaching effects on a range of sectors.
  arXiv  Detail & Related papers  (2023-10-13T09:47:24Z)
• Curriculum-Driven Edubot: A Framework for Developing Language Learning Chatbots Through Synthesizing Conversational Data [23.168347070904318]
  We present Curriculum-Driven EduBot, a framework for developing a chatbots that combines the interactive features of chatbots with the systematic material of English textbooks. We begin by extracting pertinent topics from textbooks and using large language models to generate dialogues related to these topics.
  arXiv  Detail & Related papers  (2023-09-28T19:14:18Z)
• Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems [23.093767743306973]
  Conversational tutoring systems (CTSs) offer learning experiences driven by natural language interaction. We introduce a novel type of CTS that leverages the recent advances in large language models (LLMs) in two ways. Ruffle&Riley allows a free-form conversation that follows the ITS-typical inner and outer loop structure.
  arXiv  Detail & Related papers  (2023-09-26T23:27:06Z)
• Developing Effective Educational Chatbots with ChatGPT prompts: Insights from Preliminary Tests in a Case Study on Social Media Literacy (with appendix) [43.55994393060723]
  Recent advances in language learning models with zero-shot learning capabilities, such as ChatGPT, suggest a new possibility for developing educational chatbots. We present a case study with a simple system that enables mixed-turn chatbots interactions. We examine ChatGPT's ability to pursue multiple interconnected learning objectives, adapt the educational activity to users' characteristics, such as culture, age, and level of education, and its ability to use diverse educational strategies and conversational styles.
  arXiv  Detail & Related papers  (2023-06-18T22:23:18Z)
• Towards Robust Online Dialogue Response Generation [62.99904593650087]
  We argue that this can be caused by a discrepancy between training and real-world testing. We propose a hierarchical sampling-based method consisting of both utterance-level sampling and semi-utterance-level sampling.
  arXiv  Detail & Related papers  (2022-03-07T06:51:41Z)
• Few-Shot Bot: Prompt-Based Learning for Dialogue Systems [58.27337673451943]
  Learning to converse using only a few examples is a great challenge in conversational AI. The current best conversational models are either good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL) We propose prompt-based few-shot learning which does not require gradient-based fine-tuning but instead uses a few examples as the only source of learning.
  arXiv  Detail & Related papers  (2021-10-15T14:36:45Z)
• Structural Pre-training for Dialogue Comprehension [51.215629336320305]
  We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
  arXiv  Detail & Related papers  (2021-05-23T15:16:54Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.