(AI peers) are people learning from the same standpoint: Perception of AI characters in a Collaborative Science Investigation

• URL: http://arxiv.org/abs/2506.06165v1
• Date: Fri, 06 Jun 2025 15:29:11 GMT
• Title: (AI peers) are people learning from the same standpoint: Perception of AI characters in a Collaborative Science Investigation
• Authors: Eunhye Grace Ko, Soo Hyoung Joo,
• Abstract summary: scenario-based assessment (SBA) introduces simulated agents to provide an authentic social-interactional context.<n>Recent advancements in multimodal AI, such as text-to-video technology, allow these agents to be enhanced into AI-generated characters.<n>This study investigates how learners perceive AI characters taking the role of mentor and teammates in an SBA mirroring the context of a collaborative science investigation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While the complexity of 21st-century demands has promoted pedagogical approaches to foster complex competencies, a persistent gap remains between in-class learning activities and individualized learning or assessment practices. To address this, studies have explored the use of AI-generated characters in learning and assessment. One attempt is scenario-based assessment (SBA), a technique that not only measures but also fosters the development of competencies throughout the assessment process. SBA introduces simulated agents to provide an authentic social-interactional context, allowing for the assessment of competency-based constructs while mitigating the unpredictability of real-life interactions. Recent advancements in multimodal AI, such as text-to-video technology, allow these agents to be enhanced into AI-generated characters. This mixed-method study investigates how learners perceive AI characters taking the role of mentor and teammates in an SBA mirroring the context of a collaborative science investigation. Specifically, we examined the Likert scale responses of 56 high schoolers regarding trust, social presence, and effectiveness. We analyzed the relationships between these factors and their impact on the intention to adopt AI characters through PLS-SEM. Our findings indicated that learners' trust shaped their sense of social presence with the AI characters, enhancing perceived effectiveness. Qualitative analysis further highlighted factors that foster trust, such as material credibility and alignment with learning goals, as well as the pivotal role of social presence in creating a collaborative context. This paper was accepted as an full paper for AIED 2025.

Related papers

• Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approach [0.6906005491572401]
  This study examines how users form relationships with AI-how they assess, trust, and collaborate with it in research and teaching contexts.<n>Based on 31 interviews with academics across disciplines, we developed a five-part codebook and identified five relationship types.
  arXiv  Detail & Related papers  (2025-08-02T23:41:28Z)
• AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind [2.0272430076690027]
  This study examines how students' AI literacy and prior exposure to AI technologies shape their perceptions of Socratic Mind.<n>Data from 309 undergraduates in Computer Science and Business courses were collected.
  arXiv  Detail & Related papers  (2025-07-29T10:11:24Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications [0.2812395851874055]
  This study evaluates the use of the Educational AI Hub, an AI-powered learning framework, in undergraduate civil and environmental engineering courses at a large R1 public university.<n>Students appreciated the AI assistant for its convenience and comfort, with nearly half reporting greater ease in using the AI tool.<n>While most students viewed AI use as ethically acceptable, many expressed uncertainties about institutional policies and apprehension about potential academic misconduct.
  arXiv  Detail & Related papers  (2025-06-06T03:02:49Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• Bridging the Gap: Integrating Ethics and Environmental Sustainability in AI Research and Practice [57.94036023167952]
  We argue that the efforts aiming to study AI's ethical ramifications should be made in tandem with those evaluating its impacts on the environment.<n>We propose best practices to better integrate AI ethics and sustainability in AI research and practice.
  arXiv  Detail & Related papers  (2025-04-01T13:53:11Z)
• From G-Factor to A-Factor: Establishing a Psychometric Framework for AI Literacy [1.5031024722977635]
  We establish AI literacy as a coherent, measurable construct with significant implications for education, workforce development, and social equity.<n>Study 1 revealed a dominant latent factor - termed the "A-factor" - that accounts for 44.16% of variance across diverse AI interaction tasks.<n>Study 2 refined the measurement tool by examining four key dimensions of AI literacy.<n>Regression analyses identified several significant predictors of AI literacy, including cognitive abilities (IQ), educational background, prior AI experience, and training history.
  arXiv  Detail & Related papers  (2025-03-16T14:51:48Z)
• SocialBench: Sociality Evaluation of Role-Playing Conversational Agents [85.6641890712617]
  Large language models (LLMs) have advanced the development of various AI conversational agents. SocialBench is the first benchmark designed to evaluate the sociality of role-playing conversational agents at both individual and group levels. We find that agents excelling in individual level does not imply their proficiency in group level.
  arXiv  Detail & Related papers  (2024-03-20T15:38:36Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• Analyzing Character and Consciousness in AI-Generated Social Content: A Case Study of Chirper, the AI Social Network [0.0]
  The study embarks on a comprehensive exploration of AI behavior, analyzing the effects of diverse settings on Chirper's responses. Through a series of cognitive tests, the study gauges the self-awareness and pattern recognition prowess of Chirpers. An intriguing aspect of the research is the exploration of the potential influence of a Chirper's handle or personality type on its performance.
  arXiv  Detail & Related papers  (2023-08-30T15:40:18Z)
• MAILS -- Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies [6.368014180870025]
  The questionnaire should be modular (i.e., including different facets that can be used independently of each other) to be flexibly applicable in professional life. We derived 60 items to represent different facets of AI Literacy according to Ng and colleagues conceptualisation of AI literacy. Additional 12 items to represent psychological competencies such as problem solving, learning, and emotion regulation in regard to AI.
  arXiv  Detail & Related papers  (2023-02-18T12:35:55Z)
• Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study [6.076137037890219]
  We investigate how the interaction between a human and a continually learning prediction agent develops as the agent develops competency. We develop a virtual reality environment and a time-based prediction task wherein learned predictions from a reinforcement learning (RL) algorithm augment human predictions. Our findings suggest that human trust of the system may be influenced by early interactions with the agent, and that trust in turn affects strategic behaviour.
  arXiv  Detail & Related papers  (2021-12-14T22:46:44Z)
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" [76.44130385507894]
  This paper aims to ground what we dub a 'participatory turn' in AI design by synthesizing existing literature on participation and through empirical analysis of its current practices. Based on our literature synthesis and empirical research, this paper presents a conceptual framework for analyzing participatory approaches to AI design.
  arXiv  Detail & Related papers  (2021-11-01T17:57:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.