Related papers: What Does It Take to Research with AI? A Rapid Review of Competencies to Train LLM-Literate Researchers

What Does It Take to Research with AI? A Rapid Review of Competencies to Train LLM-Literate Researchers

URL: http://arxiv.org/abs/2607.16083v1
Date: Fri, 17 Jul 2026 16:14:14 GMT
Title: What Does It Take to Research with AI? A Rapid Review of Competencies to Train LLM-Literate Researchers
Abstract summary: This rapid review analyzed 194 articles retrieved from Elicit and Google Scholar (2022 to 2025)<n>The most prevalent was domain expertise and oversight of AI outputs.<n>These results have direct implications for the design of graduate programs and AI literacy initiatives.
Score: 3.2284277337810567
License: http://creativecommons.org/licenses/by/4.0/
Abstract: The growing adoption of Large Language Models in scientific research has created a need to understand what competencies researchers and graduate students require to use these tools critically and responsibly. This rapid review analyzed 194 articles retrieved from Elicit and Google Scholar (2022 to 2025), from which 40 were selected for competency extraction and thematic analysis following independent dual screening (Gwet AC1: 0.76 to 0.83). Eight competencies were identified. The most prevalent was domain expertise and oversight of AI outputs (n = 123), encompassing subject matter mastery, systematic skepticism, source verification, and researcher accountability. Other key competencies include metacognition and decision making about AI use (n = 55), ethics and academic integrity (n = 53), prompt engineering for research (n = 38), and reproducibility of AI use (n = 29). AI literacy and technical knowledge (n = 16) was explicitly identified as a risk factor when absent, with domain expertise treated as a prerequisite for meaningful critical evaluation. The findings suggest that preparing researchers to use LLMs goes beyond technical instruction, requiring an integrated set of epistemic, ethical, and methodological competencies centered on human accountability for the knowledge produced. These results have direct implications for the design of graduate programs and AI literacy initiatives.

Related papers

ASI-Bench: At the Dawn of Artificial Superintelligence [46.937202634067766]
ASI-Bench is the first benchmark to jointly evaluate AI systems' capabilities of innovative exploration and autonomous scientific execution.<n>It contains 60 project-level research tasks across 11 scientific domains and progressively reduces methodological guidance.<n>Across 18 state-of-the-art agent--model configurations, the average score drops from 50.91 with full methodological guidance to 29.10 with only the method specified.
arXiv Detail & Related papers (2026-08-18T01:59:00Z)
A Discipline-Agnostic AI Literacy Course for Academic Research: Architecture, Pedagogy, and Implementation [0.0]
The rapid integration of generative AI into academic literacy demands curricula that equip students not only with tool proficiency but with the critical judgment to use those tools responsibly in work offerings cluster around two poles: technical AI development courses serving specialist audiences, and brief general-literacy interventions that cannot develop sustained, practice-based competencies rigorous research requires.<n>This paper reports design, rationale, and implementation of a course at Lehigh University that addresses the underserved competencies required for rigorous AI-assisted literature review.
arXiv Detail & Related papers (2026-04-29T21:47:00Z)
Report on the Scoping Workshop on AI in Science Education Research 2025 [2.0613397238713778]
Report summarizes the outcomes of a two-day international scoping workshop on the role of artificial intelligence (AI) in science education research.<n>As AI rapidly reshapes scientific practice, classroom learning, and research methods, the field faces both new opportunities and significant challenges.<n>Report concludes with actionable recommendations for training, infrastructure, and standards, along with guidance for funders, policymakers, professional organizations, and academic departments.
arXiv Detail & Related papers (2025-11-18T10:23:22Z)
The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
arXiv Detail & Related papers (2025-06-09T18:37:14Z)
From G-Factor to A-Factor: Establishing a Psychometric Framework for AI Literacy [1.5031024722977635]
We establish AI literacy as a coherent, measurable construct with significant implications for education, workforce development, and social equity.<n>Study 1 revealed a dominant latent factor - termed the "A-factor" - that accounts for 44.16% of variance across diverse AI interaction tasks.<n>Study 2 refined the measurement tool by examining four key dimensions of AI literacy.<n>Regression analyses identified several significant predictors of AI literacy, including cognitive abilities (IQ), educational background, prior AI experience, and training history.
arXiv Detail & Related papers (2025-03-16T14:51:48Z)
Generative AI Literacy: Twelve Defining Competencies [48.90506360377104]
This paper introduces a competency-based model for generative artificial intelligence (AI) literacy covering essential skills and knowledge areas necessary to interact with generative AI.<n>The competencies range from foundational AI literacy to prompt engineering and programming skills, including ethical and legal considerations.<n>These twelve competencies offer a framework for individuals, policymakers, government officials, and educators looking to navigate and take advantage of the potential of generative AI responsibly.
arXiv Detail & Related papers (2024-11-29T14:55:15Z)
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI [73.75520820608232]
We introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities.<n>These challenges encompass a wide range of disciplines spanning seven fields and 62 international Olympic competitions, rigorously examined for data leakage.<n>Our evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy, illustrating current AI limitations in complex reasoning and multimodal integration.
arXiv Detail & Related papers (2024-06-18T16:20:53Z)
AI Knowledge and Reasoning: Emulating Expert Creativity in Scientific Research [0.2209921757303168]
We introduce novel methodology that utilizes original research articles published after the AI's training cutoff. The AI are tasked with redacting findings, predicting outcomes from redacted research, and assessing prediction accuracy against reported results.
arXiv Detail & Related papers (2024-04-05T22:30:47Z)
MAILS -- Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies [6.368014180870025]
The questionnaire should be modular (i.e., including different facets that can be used independently of each other) to be flexibly applicable in professional life. We derived 60 items to represent different facets of AI Literacy according to Ng and colleagues conceptualisation of AI literacy. Additional 12 items to represent psychological competencies such as problem solving, learning, and emotion regulation in regard to AI.
arXiv Detail & Related papers (2023-02-18T12:35:55Z)
The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
arXiv Detail & Related papers (2022-12-08T23:23:39Z)
An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
arXiv Detail & Related papers (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.