Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach

• URL: http://arxiv.org/abs/2407.12687v2
• Date: Fri, 19 Jul 2024 14:03:41 GMT
• Title: Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach
• Authors: Irina Jurenka, Markus Kunesch, Kevin R. McKee, Daniel Gillick, Shaojian Zhu, Sara Wiltberger, Shubham Milind Phal, Katherine Hermann, Daniel Kasenberg, Avishkar Bhoopchand, Ankit Anand, Miruna Pîslar, Stephanie Chan, Lisa Wang, Jennifer She, Parsa Mahmoudieh, Aliya Rysbek, Wei-Jen Ko, Andrea Huber, Brett Wiltshire, Gal Elidan, Roni Rabin, Jasmin Rubinovitz, Amit Pitaru, Mac McAllister, Julia Wilkowski, David Choi, Roee Engelberg, Lidan Hackmon, Adva Levin, Rachel Griffin, Michael Sears, Filip Bar, Mia Mesar, Mana Jabbour, Arslan Chaudhry, James Cohan, Sridhar Thiagarajan, Nir Levine, Ben Brown, Dilan Gorur, Svetlana Grant, Rachel Hashimshoni, Laura Weidinger, Jieru Hu, Dawn Chen, Kuba Dolecki, Canfer Akbulut, Maxwell Bileschi, Laura Culp, Wen-Xin Dong, Nahema Marchal, Kelsie Van Deman, Hema Bajaj Misra, Michael Duah, Moran Ambar, Avi Caciularu, Sandra Lefdal, Chris Summerfield, James An, Pierre-Alexandre Kamienny, Abhinit Mohdi, Theofilos Strinopoulous, Annie Hale, Wayne Anderson, Luis C. Cobo, Niv Efron, Muktha Ananda, Shakir Mohamed, Maureen Heymans, Zoubin Ghahramani, Yossi Matias, Ben Gomes, Lila Ibrahim,
• Abstract summary: Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. We argue that this is primarily due to the difficulties with verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices. Here we present our work collaborating with learners and educators to translate high level principles from learning science into a pragmatic set of seven diverse educational benchmarks.
• Score: 25.903775277417267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major challenge facing the world is the provision of equitable and universal access to quality education. Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. The full extent of this dream, however, has not yet materialised. We argue that this is primarily due to the difficulties with verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices, reinforced by the challenges in defining excellent pedagogy. Here we present our work collaborating with learners and educators to translate high level principles from learning science into a pragmatic set of seven diverse educational benchmarks, spanning quantitative, qualitative, automatic and human evaluations; and to develop a new set of fine-tuning datasets to improve the pedagogical capabilities of Gemini, introducing LearnLM-Tutor. Our evaluations show that LearnLM-Tutor is consistently preferred over a prompt tuned Gemini by educators and learners on a number of pedagogical dimensions. We hope that this work can serve as a first step towards developing a comprehensive educational evaluation framework, and that this can enable rapid progress within the AI and EdTech communities towards maximising the positive impact of gen AI in education.

Related papers

• Transforming Teacher Education in Developing Countries: The Role of Generative AI in Bridging Theory and Practice [0.7416846035207727]
  The study focuses on Ghana, where challenges such as limited pedagogical modeling, performance-based assessments, and practitioner-expertise gaps hinder progress. GenAI has the capacity to address these issues by supporting content knowledge acquisition, a role that currently dominates teacher education programs. The study concludes by recommending empirical research to explore these roles further and develop practical steps for integrating GenAI into teacher education systems responsibly and effectively.
  arXiv  Detail & Related papers  (2024-11-16T06:46:09Z)
• From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents [78.15899922698631]
  MAIC (Massive AI-empowered Course) is a new form of online education that leverages LLM-driven multi-agent systems to construct an AI-augmented classroom. We conduct preliminary experiments at Tsinghua University, one of China's leading universities.
  arXiv  Detail & Related papers  (2024-09-05T13:22:51Z)
• Collaborative Design of AI-Enhanced Learning Activities [0.0]
  We develop a formative intervention that enables preservice teachers, in-service teachers, and EdTech specialists to effectively incorporate AI into their teaching practices. Participants reflect on AI's potential in teaching and learning by exploring different activities that can integrate AI literacy in education, including its ethical considerations and potential for innovative pedagogy.
  arXiv  Detail & Related papers  (2024-07-09T08:34:08Z)
• Exploring Teachers' Perception of Artificial Intelligence: The Socio-emotional Deficiency as Opportunities and Challenges in Human-AI Complementarity in K-12 Education [1.9797215742507548]
  In schools, teachers play a multitude of roles, serving as educators, counselors, decision-makers, and members of the school community. With recent advances in artificial intelligence (AI), there is increasing discussion about how AI can assist, complement, and collaborate with teachers. Our study seeks educators' perspectives on the potential strengths and limitations of AI across a spectrum of responsibilities.
  arXiv  Detail & Related papers  (2024-05-20T15:43:04Z)
• Enhancing Instructional Quality: Leveraging Computer-Assisted Textual Analysis to Generate In-Depth Insights from Educational Artifacts [13.617709093240231]
  We examine how artificial intelligence (AI) and machine learning (ML) methods can analyze educational content, teacher discourse, and student responses to foster instructional improvement. We identify key areas where AI/ML integration offers significant advantages, including teacher coaching, student support, and content development. This paper emphasizes the importance of aligning AI/ML technologies with pedagogical goals to realize their full potential in educational settings.
  arXiv  Detail & Related papers  (2024-03-06T18:29:18Z)
• Bringing Generative AI to Adaptive Learning in Education [58.690250000579496]
  We shed light on the intersectional studies of generative AI and adaptive learning. We argue that this union will contribute significantly to the development of the next-stage learning format in education.
  arXiv  Detail & Related papers  (2024-02-02T23:54:51Z)
• Generative AI and Its Educational Implications [0.0]
  We discuss the implications of generative AI on education across four critical sections. We propose ways in which generative AI can transform the educational landscape. Acknowledging the societal impact, we emphasize the need for updating curricula.
  arXiv  Detail & Related papers  (2023-12-26T21:29:31Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• An Experience Report of Executive-Level Artificial Intelligence Education in the United Arab Emirates [53.04281982845422]
  We present an experience report of teaching an AI course to business executives in the United Arab Emirates (UAE) Rather than focusing only on theoretical and technical aspects, we developed a course that teaches AI with a view to enabling students to understand how to incorporate it into existing business processes.
  arXiv  Detail & Related papers  (2022-02-02T20:59:53Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)
• Creation and Evaluation of a Pre-tertiary Artificial Intelligence (AI) Curriculum [58.86139968005518]
  The Chinese University of Hong Kong (CUHK)-Jockey Club AI for the Future Project (AI4Future) co-created an AI curriculum for pre-tertiary education. A team of 14 professors with expertise in engineering and education collaborated with 17 principals and teachers from 6 secondary schools to co-create the curriculum. The co-creation process generated a variety of resources which enhanced the teachers knowledge in AI, as well as fostered teachers autonomy in bringing the subject matter into their classrooms.
  arXiv  Detail & Related papers  (2021-01-19T11:26:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.