Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS)

• URL: http://arxiv.org/abs/2607.14301v1
• Date: Wed, 15 Jul 2026 19:04:11 GMT
• Title: Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS)
• Abstract summary: How students rely on generative AI (GenAI) has become a central question for learning, academic integrity, and educational equity.<n>This study developed and validated the GenAI Reliance Types Scale (GenAI-RTS), a 20-item instrument measuring four theoretically derived types of GenAI reliance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As generative AI (GenAI) becomes increasingly embedded in undergraduate academic writing, how students rely on these tools, rather than simply whether they use them, has become a central question for learning, academic integrity, and educational equity. Existing measures of reliance were developed inductively, focused on discrete problem-solving tasks, and validated mainly with homogeneous samples. This study developed and validated the GenAI Reliance Types Scale (GenAI-RTS), a 20-item instrument measuring four theoretically derived types of GenAI reliance: Strategic, Instrumental, Dependent, and Dialogic. Validation followed the multisource framework of the Standards for Educational and Psychological Testing, drawing on a survey of 382 undergraduates at a U.S. Minority-Serving Institution and interviews with 14 purposively sampled students. Confirmatory factor analyses of six competing models supported a five-factor structure in which Strategic Reliance comprises two facets, Deliberate Use and Critical Evaluation, alongside Instrumental, Dependent, and Dialogic factors (CFI = .92, RMSEA = .08; DWLS CFI = .98, RMSEA = .07). Subscale reliability was acceptable to good (omega = .75-.88), and scalar measurement invariance held across gender, first-generation status, and STEM/non-STEM majors, to our knowledge the first such evidence for a GenAI reliance instrument. Rasch analysis indicated that a five-point response format would improve category functioning. Strategic reliance was positively associated with AI literacy, and the reliance types differentiated students across multiple writing process and outcome variables. The GenAI-RTS offers researchers and educators a theoretically grounded, psychometrically validated instrument for identifying undergraduate reliance profiles and supporting research, assessment, and AI literacy intervention.

Related papers

• GenAIT: Development and Validation of an Objective Generative AI Literacy Test for High School Students [0.5735035463793009]
  There is growing international interest in generative AI (GenAI) literacy and its assessment among high school students, but objective assessment in this population remains underdeveloped.<n>This article reports the iterative development and validation of the GenAI Literacy Test (GenAIT), an 18-item multiple-choice test measuring high school students' conceptual knowledge about GenAI.
  arXiv  Detail & Related papers  (2026-08-26T13:58:56Z)
• Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers: A Mixed-Methods Study at a Minority-Serving R1 University [0.0]
  Existing instruments assess reliance solely by frequency of use.<n>Students' value and cost beliefs predicted the intensity of their reliance on large language models.<n> Strategic users, those who engaged AI most deliberately, scored lowest on standard outcome measures.
  arXiv  Detail & Related papers  (2026-06-27T05:56:03Z)
• The GenAI Skill Bypass: Mapping Divergent Pathways of University Students and Staff AI Literacy [0.9039347265533652]
  Higher education institutions are increasingly expected to ensure that both students and staff develop Generative AI (GenAI)<n>Current educational frameworks typically assume a linear progression of GenAI literacy.<n>This paper challenges such an assumption through a psychometric analysis of a taxonomy-based self-assessment instrument.
  arXiv  Detail & Related papers  (2026-06-10T06:53:39Z)
• Profiles of AI Dependency: A Latent Class Analysis of Filipino Students' Academic Competencies [0.0]
  The increasing dependency among Filipino college students on artificial intelligence (AI) poses concerns about the potential decline of fundamental academic competencies.<n>This study examines the extent of AI dependency and its perceived effects on students' critical thinking, writing skills, learning independence, research skills, and academic engagement.
  arXiv  Detail & Related papers  (2026-04-30T02:59:52Z)
• AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot [45.90272517740283]
  We report the first large-scale field deployment of AI-assisted peer review.<n>Every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system.
  arXiv  Detail & Related papers  (2026-04-15T14:51:07Z)
• Exploring Student Perception on Gen AI Adoption in Higher Education: A Descriptive Study [1.978884131103313]
  This study examines how students perceive, use, and evaluate Generative Artificial Intelligence (GenAI)<n>Data were collected through a questionnaire administered to 436 postgraduate Computer Science students at the University of Hertfordshire.
  arXiv  Detail & Related papers  (2026-03-29T17:19:39Z)
• AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite [75.58737079136942]
  We present AstaBench, a suite that provides the first holistic measure of agentic ability to perform scientific research.<n>Our suite comes with the first scientific research environment with production-grade search tools.<n>Our evaluation of 57 agents across 22 agent classes reveals several interesting findings.
  arXiv  Detail & Related papers  (2025-10-24T17:10:26Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Synergizing Self-Regulation and Artificial-Intelligence Literacy Towards Future Human-AI Integrative Learning [92.34299949916134]
  Self-regulated learning (SRL) and Artificial-Intelligence (AI) literacy are becoming key competencies for successful human-AI interactive learning.<n>This study analyzed data from 1,704 Chinese undergraduates using clustering methods to uncover four learner groups.
  arXiv  Detail & Related papers  (2025-03-31T13:41:21Z)
• Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge [78.35388859345056]
  We argue that the ML community would benefit from learning from and drawing on the social sciences when developing measurement instruments for evaluating GenAI systems.<n>We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI systems.
  arXiv  Detail & Related papers  (2025-02-01T21:09:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.