Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education

• URL: http://arxiv.org/abs/2603.20220v1
• Date: Tue, 03 Mar 2026 23:20:33 GMT
• Title: Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education
• Abstract summary: Existing AI literacy instruments, however, primarily target the general public, students, or K-12 teachers.<n>This study develops and validates the Faculty Artificial Intelligence Literacy and competency (FALCON-AI) Scale.
• Score: 0.27998963147546146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of artificial intelligence (AI) in higher education underscores the growing importance of faculty AI literacy and competency across teaching, research, and service. Existing AI literacy instruments, however, primarily target the general public, students, or K-12 teachers, and therefore lack the role-embedded indicators and psychometric validation needed for scalable assessment among university faculty. Grounded in the Critical Tech-resilient Literacies (CTRL) framework, this study develops and validates the Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale as a concise and practically deployable tool for higher education contexts. Using a theory-driven development process, we generated an initial pool of 43 items mapped to three literacies (functional, evaluative, and ethical literacy) and situated them across four faculty work domains (general, teaching, research, service/administration), creating a 3 x 4 framework. Content validation was conducted through structured interviews with four subject-matter experts, supplemented by a GPT-based reviewer to triangulate ratings of clarity, relevance, and necessity, yielding refined 39 items for pilot testing. Pilot testing involved 269 valid responses, which were analyzed using confirmatory factor analysis (CFA). CFA evaluated the theoretically specified structure, followed by item reduction to minimize respondent burden while preserving content coverage. The final 23-item FALCON-AI demonstrated good model fit for the AI Literacy x Faculty Work measurement and strong reliability. This study presents a validated FALCON-AI scale with good reliability and validity, offering a refined practical instrument for assessing faculty AI in higher education.

Related papers

• EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models [66.28519000191191]
  EmoSBench is the first comprehensive EI evaluation benchmark for Spoken Language Models.<n>EmoS reaches 83.8% accuracy, approaching human-level performance.
  arXiv  Detail & Related papers  (2026-08-10T06:58:30Z)
• AI-Automation Tooling in Computer Engineering Education: Mixed-Methods TAM/UTAUT Evidence for a General Acceptance Attitude [0.0]
  This paper reports a mixed-methods, cross-sectional study of undergraduate computer engineering students' acceptance of AI automation tooling.<n> Acceptance was favorable across all six constructs with large effect sizes.<n>Findings support the curricular adoption of AI automation tooling in undergraduate computing education.
  arXiv  Detail & Related papers  (2026-05-10T11:00:56Z)
• A Discipline-Agnostic AI Literacy Course for Academic Research: Architecture, Pedagogy, and Implementation [0.0]
  The rapid integration of generative AI into academic literacy demands curricula that equip students not only with tool proficiency but with the critical judgment to use those tools responsibly in work offerings cluster around two poles: technical AI development courses serving specialist audiences, and brief general-literacy interventions that cannot develop sustained, practice-based competencies rigorous research requires.<n>This paper reports design, rationale, and implementation of a course at Lehigh University that addresses the underserved competencies required for rigorous AI-assisted literature review.
  arXiv  Detail & Related papers  (2026-04-29T21:47:00Z)
• Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models: Performance Benchmarking and Reasoning-Based Prompting Strategies [6.934935343001595]
  We extend prior curricular analytics research by examining a broader range of curriculum documents, competency frameworks, and models.<n>Using 7,600 manually annotated curriculum-competency alignment scores, we assess the informativeness of different curriculum sources.<n>We introduce a reasoning-based prompting strategy, Curricular CoT, to strengthen LLMs' pedagogical reasoning.
  arXiv  Detail & Related papers  (2026-01-16T04:07:23Z)
• How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures [6.943056322842481]
  The widespread adoption of Artificial Intelligence (AI) in K-12 education highlights the need for psychometrically-tested measures of teachers' AI literacy.<n>This study developed and evaluated SR and OB measures of teacher AI literacy within the established framework of Concept, Use, Evaluate, and Ethics.<n>Latent profile analysis identified six distinct profiles, including overestimation (SR > OB), underestimation (SR OB), alignment (SR close to OB) and a unique low-SR/low-OB profile among teachers without AI literacy experience.
  arXiv  Detail & Related papers  (2026-01-03T00:10:41Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• Decoding Instructional Dialogue: Human-AI Collaborative Analysis of Teacher Use of AI Tool at Scale [9.092920230987684]
  The integration of large language models into educational tools has the potential to substantially impact how teachers plan instruction.<n>This paper presents a human-AI collaborative methodology for large-scale qualitative analysis of over 140,000 educator-AI messages.
  arXiv  Detail & Related papers  (2025-07-23T23:23:38Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• From G-Factor to A-Factor: Establishing a Psychometric Framework for AI Literacy [1.5031024722977635]
  We establish AI literacy as a coherent, measurable construct with significant implications for education, workforce development, and social equity.<n>Study 1 revealed a dominant latent factor - termed the "A-factor" - that accounts for 44.16% of variance across diverse AI interaction tasks.<n>Study 2 refined the measurement tool by examining four key dimensions of AI literacy.<n>Regression analyses identified several significant predictors of AI literacy, including cognitive abilities (IQ), educational background, prior AI experience, and training history.
  arXiv  Detail & Related papers  (2025-03-16T14:51:48Z)
• STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond [68.47402386668846]
  We introduce Structured Reasoning In Critical Text Assessment (STRICTA) to model text assessment as an explicit, step-wise reasoning process.<n>STRICTA breaks down the assessment into a graph of interconnected reasoning steps drawing on causality theory.<n>We apply STRICTA to a dataset of over 4000 reasoning steps from roughly 40 biomedical experts on more than 20 papers.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Explainable Authorship Identification in Cultural Heritage Applications: Analysis of a New Perspective [48.031678295495574]
  We explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId. In particular, we assess the relative merits of three different types of XAI techniques on three different AId tasks. Our analysis shows that, while these techniques make important first steps towards explainable Authorship Identification, more work remains to be done.
  arXiv  Detail & Related papers  (2023-11-03T20:51:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.