Analyzing Middle School Students' Dialogue and Behaviors during Collaborative AI Chatbot Development Using Ordered Network Analysis

• URL: http://arxiv.org/abs/2607.21603v1
• Date: Fri, 15 May 2026 16:20:06 GMT
• Title: Analyzing Middle School Students' Dialogue and Behaviors during Collaborative AI Chatbot Development Using Ordered Network Analysis
• Abstract summary: This study examines collaborative interaction among middle school students engaged in the design and development of an AI chatbots.<n>Results reveal that higher-quality chatbots are associated with more integrated sequences linking explanation, testing, and refinement.
• Score: 7.734887241066237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Artificial Intelligence (AI) education has become a key component of K-12 curricula, activities such as designing and developing conversational agents are increasingly used as instructional practice. Prior work has primarily examined these activities by focusing on students' learning outcomes or the quality of final AI artifacts, offering limited insight into the collaborative processes through which learning unfolds during AI system development. Although the AIED community has a long history of studying collaborative learning in STEM and Computing education, the emergence of AI learning environments in which students build AI systems presents new opportunities to understand how collaboration unfolds in AI education contexts. Grounded in these foundational works, the current study examines collaborative interaction among middle school students engaged in the design and development of an AI chatbot. Using Ordered Network Analysis of students' dialogue and development actions, we characterize how collaboration is organized over time and how interaction patterns relate to chatbot quality and AI knowledge outcomes. Results reveal that higher-quality chatbots are associated with more integrated sequences linking explanation, testing, and refinement. Interaction patterns involving articulated reasoning and repeated testing and revision in response to chatbot output were also associated with stronger AI knowledge outcomes. These findings provide a process-oriented account of collaborative AI chatbot development and extend AIED research on collaborative learning processes to AI education contexts.

Related papers

• Where's the Structure? A Systematic Literature Review of Empirical Research on Human-AI Collaboration and Hybrid Intelligence for Learning [0.0]
  "Human-AI collaboration" is where human(s) and AI components interact to promote human learning.<n> unstructured interaction does not necessarily produce an effective learning experience.<n>This paper reports a systematic literature review of empirical studies on human-AI collaboration and hybrid intelligence for learning support.
  arXiv  Detail & Related papers  (2026-05-30T07:54:32Z)
• Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery [72.49593499512092]
  We identify and characterize a distinct workflow we term intentmaking.<n>We frame this as a natural extension to sensemaking.<n>Our documentation of these themes suggests an approach to designing AI tools for scientific discovery.
  arXiv  Detail & Related papers  (2026-05-07T09:30:25Z)
• AI Combines, Humans Socialise: A SECI-based Experience Report on Business Simulation Games [0.0]
  This paper reports on the integration of generative AI tools into a Business Simulation Games (BSG) designed for engineering students.<n> AI was embedded as a support mechanism during the simulation to assist students in analysing events, reformulating information, and generating decision-relevant insights.<n>The results suggest a functional boundary in human-AI collaboration within simulation-based learning.
  arXiv  Detail & Related papers  (2026-02-24T07:26:16Z)
• AI in data science education: experiences from the classroom [0.0]
  This study explores the integration of AI, particularly large language models (LLMs) like ChatGPT, into educational settings.<n>Interviews with course coordinators from data science courses at Wageningen University identify both the benefits and challenges associated with AI in the classroom.<n>Study highlights the importance of responsible AI usage, ethical considerations, and the need for adapting assessment methods to ensure educational outcomes are met.
  arXiv  Detail & Related papers  (2025-10-01T11:45:25Z)
• Look: AI at Work! - Analysing Key Aspects of AI-support at the Work Place [39.146761527401424]
  We look at the areas of AI that the applications are concerned with.<n>This includes the importance of high-quality data for training learning-based systems.<n>In terms of the psychological factors we derive research questions to investigate in the development of AI supported work systems.
  arXiv  Detail & Related papers  (2025-09-02T12:51:23Z)
• Let people fail! Exploring the influence of explainable virtual and robotic agents in learning-by-doing tasks [45.23431596135002]
  This study compares the effects of classic vs. partner-aware explanations on human behavior and performance during a learning-by-doing task. Results indicated that partner-aware explanations influenced participants differently based on the type of artificial agents involved.
  arXiv  Detail & Related papers  (2024-11-15T13:22:04Z)
• Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students [35.05435052195561]
  The increasing use of Artificial Intelligence (AI) by students in learning presents new challenges for assessing their learning outcomes. This paper introduces a co-design study to explore the potential of students' AI usage data as a novel material for assessment.
  arXiv  Detail & Related papers  (2024-01-26T14:49:29Z)
• Collaborative Learning with Artificial Intelligence Speakers (CLAIS): Pre-Service Elementary Science Teachers' Responses to the Prototype [0.5113447003407372]
  The CLAIS system is designed to have 3-4 human learners join an AI speaker to form a small group, where humans and AI are considered as peers participating in the Jigsaw learning process. The CLAIS system was successfully implemented in a Science Education course session with 15 pre-service elementary science teachers.
  arXiv  Detail & Related papers  (2023-12-20T01:19:03Z)
• Toward enriched Cognitive Learning with XAI [44.99833362998488]
  We introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by artificial intelligence (AI) tools. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle problems to enhance problem-solving skills.
  arXiv  Detail & Related papers  (2023-12-19T16:13:47Z)
• AI Chatbots as Multi-Role Pedagogical Agents: Transforming Engagement in CS Education [8.898863361318817]
  We develop, implement, and evaluate a novel learning environment enriched with four distinct chatbots. These roles cater to the three innate psychological needs of learners - competence, autonomy, and relatedness. The system embraces an inquiry-based learning paradigm, encouraging students to ask questions, seek solutions, and explore their curiosities.
  arXiv  Detail & Related papers  (2023-08-08T02:13:44Z)
• Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback [42.19685958922537]
  We argue that human-AI collaboration should be interactive, with humans monitoring the work of AI agents and providing feedback that the agent can understand and utilize. In this work, we explore these directions using the challenging task defined by the IGLU competition, an interactive grounded language understanding task in a MineCraft-like world.
  arXiv  Detail & Related papers  (2023-04-21T05:37:59Z)
• Human-Robot Collaboration and Machine Learning: A Systematic Review of Recent Research [69.48907856390834]
  Human-robot collaboration (HRC) is the approach that explores the interaction between a human and a robot. This paper proposes a thorough literature review of the use of machine learning techniques in the context of HRC.
  arXiv  Detail & Related papers  (2021-10-14T15:14:33Z)
• Learning and Reasoning for Robot Dialog and Navigation Tasks [44.364322669414776]
  We develop algorithms for robot task completions, while looking into the complementary strengths of reinforcement learning and probabilistic reasoning techniques. The robots learn from trial-and-error experiences to augment their declarative knowledge base. We have implemented and evaluated the developed algorithms using mobile robots conducting dialog and navigation tasks.
  arXiv  Detail & Related papers  (2020-05-20T03:20:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.