The Agency Gap: How Generative AI Literacy Shapes Independent Writing after AI Support

• URL: http://arxiv.org/abs/2507.04398v2
• Date: Thu, 16 Oct 2025 17:26:46 GMT
• Title: The Agency Gap: How Generative AI Literacy Shapes Independent Writing after AI Support
• Authors: Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan,
• Abstract summary: This study investigates the extent to which GenAI literacy predicts student writing performance in contexts that require self-initiation and regulation.<n>Writing was evaluated across insightfulness, visual data integration, organisation, linguistic quality, and critical thinking.
• Score: 4.013708138746962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI (GenAI) tools are rapidly transforming higher education, yet little is known about how students' GenAI literacy shapes their ability to perform independently once such support is removed. This study investigates what we term the agency gap, introduced as the extent to which GenAI literacy predicts student writing performance in contexts that require self-initiation and regulation. Seventy-nine medical and nursing students completed multimodal academic writing tasks based on visual data, supported either by a reactive or proactive GenAI chatbot, followed by a parallel task without AI support. Writing was evaluated across insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Results showed that GenAI literacy predicted independent writing performance only in the reactive condition, where students had to actively mobilise their own strategies. Mediation analyses revealed no indirect effect via in-task performance, indicating that GenAI literacy acts as a direct, task-general competence rather than a proxy for domain knowledge or other literacies. By contrast, proactive scaffolding equalised outcomes across literacy levels, reducing reliance on learners' GenAI literacy. The agency gap highlights when GenAI literacy matters most, with implications for designing equitable AI-supported learning environments that both leverage and mitigate differences in students' GenAI literacy.

Related papers

• AI Literacy, Safety Awareness, and STEM Career Aspirations of Australian Secondary Students: Evaluating the Impact of Workshop Interventions [38.350232667249095]
  Deepfakes and other forms of synthetic media pose growing safety risks for adolescents.<n>This study evaluates the impact of Day of AI Australia's workshop-based intervention on Australian secondary students.
  arXiv  Detail & Related papers  (2026-01-30T02:55:53Z)
• Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming [3.2383459424229835]
  This study investigates how GenAI assistance compares to conventional online resources in supporting knowledge gains across different proficiency levels.<n>We conducted a controlled user experiment with 24 undergraduate students of two different levels of programming experience.<n>Our findings reveal that generating complete solutions with GenAI significantly improves task performance, especially for beginners, but does not consistently result in knowledge gains.
  arXiv  Detail & Related papers  (2025-11-17T11:42:24Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues [48.99818550820575]
  Recent studies have shown that strategies used by tutors can have significant effects on student outcomes.<n>Few works have studied predicting tutor strategy in dialogues.<n>We investigate the ability of modern LLMs, particularly Llama 3 and GPT-4o, to predict both future tutor moves and student outcomes in dialogues.
  arXiv  Detail & Related papers  (2025-07-09T14:47:35Z)
• AI in the Writing Process: How Purposeful AI Support Fosters Student Writing [0.3641292357963815]
  The ubiquity of technologies like ChatGPT has raised concerns about their impact on student writing.<n>This paper investigates how different AI support approaches affect writers' sense of agency and depth of knowledge transformation.
  arXiv  Detail & Related papers  (2025-06-25T16:34:09Z)
• Integrating Universal Generative AI Platforms in Educational Labs to Foster Critical Thinking and Digital Literacy [0.3749861135832073]
  This paper presents a new educational framework for integrating generative artificial intelligence (GenAI) platforms into laboratory activities.<n> Recognizing the limitations and risks of uncritical reliance on large language models (LLMs), the proposed pedagogical model reframes GenAI as a research subject and cognitive tool.
  arXiv  Detail & Related papers  (2025-06-11T17:45:51Z)
• "We need to avail ourselves of GenAI to enhance knowledge distribution": Empowering Older Adults through GenAI Literacy [0.49157446832511503]
  Older adults often exhibit greater reservations about adopting emerging technologies.<n>This study examines strategies for delivering GenAI literacy to older adults.<n> Quantitative data indicated a trend toward improved AI literacy, though the results were not statistically significant.
  arXiv  Detail & Related papers  (2025-06-06T16:38:37Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• Human-AI Collaboration or Academic Misconduct? Measuring AI Use in Student Writing Through Stylometric Evidence [0.0]
  This research investigates the use of authorship verification (AV) techniques to quantify AI assistance in academic writing.<n>We use three datasets - including a public dataset (PAN-14) and two from University of Melbourne students from various courses.<n>We develop an adapted Feature Vector Difference AV methodology to construct robust academic writing profiles for students.
  arXiv  Detail & Related papers  (2025-05-13T00:36:36Z)
• Evaluating the AI-Lab Intervention: Impact on Student Perception and Use of Generative AI in Early Undergraduate Computer Science Courses [0.0]
  Generative AI (GenAI) is rapidly entering computer science education.<n>Concerns about overreliance coexist with a gap in research on structured scaffolding to guide tool use in formal courses.<n>This study examines the impact of a dedicated "AI-Lab" intervention on undergraduate students.
  arXiv  Detail & Related papers  (2025-04-30T18:12:42Z)
• Synergizing Self-Regulation and Artificial-Intelligence Literacy Towards Future Human-AI Integrative Learning [92.34299949916134]
  Self-regulated learning (SRL) and Artificial-Intelligence (AI) literacy are becoming key competencies for successful human-AI interactive learning.<n>This study analyzed data from 1,704 Chinese undergraduates using clustering methods to uncover four learner groups.
  arXiv  Detail & Related papers  (2025-03-31T13:41:21Z)
• Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners [3.2668433085737036]
  Students can compose texts by integrating their own words with AI-generated text.<n>This study investigated how 59 Hong Kong secondary school students interacted with AI-generated text to compose a feature article.
  arXiv  Detail & Related papers  (2025-03-01T06:29:00Z)
• AI Literacy in K-12 and Higher Education in the Wake of Generative AI: An Integrative Review [3.5297361401370044]
  There is little consensus among researchers and practitioners on how to discuss and design AI literacy interventions.<n>This paper applies an integrative review method to examine empirical and theoretical AI literacy studies published since 2020.
  arXiv  Detail & Related papers  (2025-02-27T23:32:03Z)
• Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023 [38.83649319653387]
  There is limited prior research on computing students' use and perceptions of GenAI. We surveyed all computer science majors in a small engineering-focused R1 university. We discuss the impact of our findings on the emerging conversation around GenAI and education.
  arXiv  Detail & Related papers  (2024-11-17T20:17:47Z)
• Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI [41.96102438774773]
  This work presents the findings from a university-level competition, which challenged participants to design prompts for eliciting biased outputs from GenAI tools. We quantitatively and qualitatively analyze the competition submissions and identify a diverse set of biases in GenAI and strategies employed by participants to induce bias in GenAI.
  arXiv  Detail & Related papers  (2024-10-20T18:44:45Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration [39.96202614397779]
  Recent research has explored the potential for artificial intelligence to support and augment humans in data storytelling. There lacks a systematic review to understand data storytelling tools from the perspective of human-AI collaboration. This paper investigated existing tools with a framework from two perspectives: the stages in the storytelling workflow where a tool serves, including analysis, planning, implementation, and communication, and the roles of humans and AI.
  arXiv  Detail & Related papers  (2023-09-27T15:30:50Z)
• The Impact of Artificial Intelligence on the Evolution of Digital Education: A Comparative Study of OpenAI Text Generation Tools including ChatGPT, Bing Chat, Bard, and Ernie [0.17999333451993949]
  This review paper delves deep into the rapidly evolving landscape of digital education by contrasting the capabilities and impact of OpenAI's pioneering text generation tools like Bing Chat, Bard, Ernie.<n>The study underscores its role in democratizing education, fostering autodidacticism, and magnifying student engagement.<n>However, with such transformative power comes the potential for misuse, as text-generation tools can inadvertently challenge academic integrity.
  arXiv  Detail & Related papers  (2023-09-05T08:15:00Z)
• Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption [0.0]
  We introduce "AI-Lab," a framework for guiding students in effectively leveraging GenAI within core programming courses. By identifying and rectifying GenAI's errors, students enrich their learning process. For educators, AI-Lab provides mechanisms to explore students' perceptions of GenAI's role in their learning experience.
  arXiv  Detail & Related papers  (2023-08-23T17:20:37Z)
• Exploring User Perspectives on ChatGPT: Applications, Perceptions, and Implications for AI-Integrated Education [40.38809129759498]
  ChatGPT is most commonly used in the domains of higher education, K-12 education, and practical skills training. On one hand, some users view it as a transformative tool capable of amplifying student self-efficacy and learning motivation. On the other hand, there is a degree of apprehension among concerned users.
  arXiv  Detail & Related papers  (2023-05-22T15:13:14Z)
• AGI: Artificial General Intelligence for Education [41.45039606933712]
  This position paper reviews artificial general intelligence (AGI)'s key concepts, capabilities, scope, and potential within future education. It highlights that AGI can significantly improve intelligent tutoring systems, educational assessment, and evaluation procedures. The paper emphasizes that AGI's capabilities extend to understanding human emotions and social interactions.
  arXiv  Detail & Related papers  (2023-04-24T22:31:59Z)
• Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task [32.6507926764587]
  Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software. We show that deep reinforcement learning can be used to provide adaptive pedagogical support to students learning about the concept of volume.
  arXiv  Detail & Related papers  (2023-04-11T02:11:24Z)
• StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement [61.47157418485633]
  We developed StoryBuddy, an AI-enabled system for parents to create interactive storytelling experiences. A user study validated StoryBuddy's usability and suggested design insights for future parent-AI collaboration systems.
  arXiv  Detail & Related papers  (2022-02-13T04:53:28Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.