Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing

• URL: http://arxiv.org/abs/2507.04398v1
• Date: Sun, 06 Jul 2025 14:01:06 GMT
• Title: Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing
• Authors: Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan,
• Abstract summary: Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments.<n>While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students' GenAI literacy influences their independent multimodal writing skills.<n>This study examined 79 higher education students' multimodal academic writing performance using a comparative research design.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments. While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students' GenAI literacy influences their independent multimodal writing skills or how chatbot interaction strategies (passive reactive vs. proactive scaffolding) impact learning. This study examined 79 higher education students' multimodal academic writing performance using a comparative research design. Students completed writing tasks integrating visual data under two chatbot-assisted conditions (passive vs. proactive) and subsequently without AI assistance. Their writing performance was rigorously evaluated across five dimensions, including insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Ordinal logistic regression and correlation analyses revealed that higher levels of GenAI literacy significantly predicted stronger independent multimodal writing performance immediately after AI assistance removal, particularly for students using passive chatbots requiring active prompting. These results highlight the critical role of GenAI literacy and specific chatbot interaction strategies in shaping students' capacities for independent multimodal academic writing. Our findings emphasise the need for purposeful integration of GenAI literacy training into curricula and balancing external scaffolding support with autonomous learning opportunities. This research offers valuable recommendations for educators leveraging AI-enhanced pedagogies to optimise student writing outcomes and technological engagement strategies.

Related papers

• Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues [48.99818550820575]
  Recent studies have shown that strategies used by tutors can have significant effects on student outcomes.<n>Few works have studied predicting tutor strategy in dialogues.<n>We investigate the ability of modern LLMs, particularly Llama 3 and GPT-4o, to predict both future tutor moves and student outcomes in dialogues.
  arXiv  Detail & Related papers  (2025-07-09T14:47:35Z)
• AI in the Writing Process: How Purposeful AI Support Fosters Student Writing [0.3641292357963815]
  The ubiquity of technologies like ChatGPT has raised concerns about their impact on student writing.<n>This paper investigates how different AI support approaches affect writers' sense of agency and depth of knowledge transformation.
  arXiv  Detail & Related papers  (2025-06-25T16:34:09Z)
• Integrating Universal Generative AI Platforms in Educational Labs to Foster Critical Thinking and Digital Literacy [0.3749861135832073]
  This paper presents a new educational framework for integrating generative artificial intelligence (GenAI) platforms into laboratory activities.<n> Recognizing the limitations and risks of uncritical reliance on large language models (LLMs), the proposed pedagogical model reframes GenAI as a research subject and cognitive tool.
  arXiv  Detail & Related papers  (2025-06-11T17:45:51Z)
• Human-AI Collaboration or Academic Misconduct? Measuring AI Use in Student Writing Through Stylometric Evidence [0.0]
  This research investigates the use of authorship verification (AV) techniques to quantify AI assistance in academic writing.<n>We use three datasets - including a public dataset (PAN-14) and two from University of Melbourne students from various courses.<n>We develop an adapted Feature Vector Difference AV methodology to construct robust academic writing profiles for students.
  arXiv  Detail & Related papers  (2025-05-13T00:36:36Z)
• Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners [3.2668433085737036]
  Students can compose texts by integrating their own words with AI-generated text.<n>This study investigated how 59 Hong Kong secondary school students interacted with AI-generated text to compose a feature article.
  arXiv  Detail & Related papers  (2025-03-01T06:29:00Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration [39.96202614397779]
  Recent research has explored the potential for artificial intelligence to support and augment humans in data storytelling. There lacks a systematic review to understand data storytelling tools from the perspective of human-AI collaboration. This paper investigated existing tools with a framework from two perspectives: the stages in the storytelling workflow where a tool serves, including analysis, planning, implementation, and communication, and the roles of humans and AI.
  arXiv  Detail & Related papers  (2023-09-27T15:30:50Z)
• The Impact of Artificial Intelligence on the Evolution of Digital Education: A Comparative Study of OpenAI Text Generation Tools including ChatGPT, Bing Chat, Bard, and Ernie [0.17999333451993949]
  This review paper delves deep into the rapidly evolving landscape of digital education by contrasting the capabilities and impact of OpenAI's pioneering text generation tools like Bing Chat, Bard, Ernie.<n>The study underscores its role in democratizing education, fostering autodidacticism, and magnifying student engagement.<n>However, with such transformative power comes the potential for misuse, as text-generation tools can inadvertently challenge academic integrity.
  arXiv  Detail & Related papers  (2023-09-05T08:15:00Z)
• Exploring User Perspectives on ChatGPT: Applications, Perceptions, and Implications for AI-Integrated Education [40.38809129759498]
  ChatGPT is most commonly used in the domains of higher education, K-12 education, and practical skills training. On one hand, some users view it as a transformative tool capable of amplifying student self-efficacy and learning motivation. On the other hand, there is a degree of apprehension among concerned users.
  arXiv  Detail & Related papers  (2023-05-22T15:13:14Z)
• AGI: Artificial General Intelligence for Education [41.45039606933712]
  This position paper reviews artificial general intelligence (AGI)'s key concepts, capabilities, scope, and potential within future education. It highlights that AGI can significantly improve intelligent tutoring systems, educational assessment, and evaluation procedures. The paper emphasizes that AGI's capabilities extend to understanding human emotions and social interactions.
  arXiv  Detail & Related papers  (2023-04-24T22:31:59Z)
• Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task [32.6507926764587]
  Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software. We show that deep reinforcement learning can be used to provide adaptive pedagogical support to students learning about the concept of volume.
  arXiv  Detail & Related papers  (2023-04-11T02:11:24Z)
• StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement [61.47157418485633]
  We developed StoryBuddy, an AI-enabled system for parents to create interactive storytelling experiences. A user study validated StoryBuddy's usability and suggested design insights for future parent-AI collaboration systems.
  arXiv  Detail & Related papers  (2022-02-13T04:53:28Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.