Leveraging Virtual Reality and AI Tutoring for Language Learning: A Case Study of a Virtual Campus Environment with OpenAI GPT Integration with Unity 3D

• URL: http://arxiv.org/abs/2411.12619v1
• Date: Tue, 19 Nov 2024 16:26:19 GMT
• Title: Leveraging Virtual Reality and AI Tutoring for Language Learning: A Case Study of a Virtual Campus Environment with OpenAI GPT Integration with Unity 3D
• Authors: Adithya TG, Abhinavaram N, Gowri Srinivasa,
• Abstract summary: We have developed a scenario which has a virtual campus environment using Unity. Within this virtual environment, we have an AI tutor powered by OpenAI's GPT model. This provided language learning support in Hindi, as GPT is able to take care of language translation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a new approach to multiple language learning, with Hindi the language to be learnt in our case, by using the integration of virtual reality environments and AI enabled tutoring systems using OpenAIs GPT api calls. We have developed a scenario which has a virtual campus environment using Unity which focuses on a detailed representation of our universitys buildings 11th floor, where most of the cultural and technological activities take place. Within this virtual environment that we have created, we have an AI tutor powered by OpenAI's GPT model which was called using an api which moves around with the user. This provided language learning support in Hindi, as GPT is able to take care of language translation. Our approach mainly involves utilising speech to text, text to text conversion and text to speech capabilities to facilitate real time interaction between users and the AI tutor in the presence of internet. This research demonstrates the use of combining VR technology with AI tutoring for immersive language learning experiences and provides interaction.

Related papers

• VTutor: An Open-Source SDK for Generative AI-Powered Animated Pedagogical Agents with Multi-Media Output [10.419430731115405]
  This paper introduces VTutor, an open-source Software Development Kit (SDK) that combines generative AI with advanced animation technologies. VTutor enables researchers and developers to design emotionally resonant, contextually adaptive learning agents. This toolkit enhances learner engagement, feedback receptivity, and human-AI interaction while promoting trustworthy AI principles in education.
  arXiv  Detail & Related papers  (2025-02-06T14:27:54Z)
• Generative AI, Pragmatics, and Authenticity in Second Language Learning [0.0]
  There are obvious benefits to integrating generative AI (artificial intelligence) into language learning and teaching. However, due to how AI systems under-stand human language, they lack the lived experience to be able to use language with the same social awareness as humans. There are built-in linguistic and cultural biases based on their training data which is mostly in English and predominantly from Western sources.
  arXiv  Detail & Related papers  (2024-10-18T11:58:03Z)
• Multimodal 3D Fusion and In-Situ Learning for Spatially Aware AI [10.335943413484815]
  seamless integration of virtual and physical worlds in augmented reality benefits from the system semantically "understanding" the physical environment. We introduce a multimodal 3D object representation that unifies both semantic and linguistic knowledge with the geometric representation. We demonstrate the usefulness of the proposed system through two real-world AR applications on Magic Leap 2: a) spatial search in physical environments with natural language and b) an intelligent inventory system that tracks object changes over time.
  arXiv  Detail & Related papers  (2024-10-06T23:25:21Z)
• VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications [2.5022287664959446]
  This study introduces a pioneering approach utilizing Visual Language Models within VR environments to enhance user interaction and task efficiency. Our system facilitates real-time, intuitive user interactions through natural language processing, without relying on visual text instructions.
  arXiv  Detail & Related papers  (2024-05-19T12:56:00Z)
• Distributed agency in second language learning and teaching through generative AI [0.0]
  ChatGPT can provide informal second language practice through chats in written or voice forms. Instructors can use AI to build learning and assessment materials in a variety of media.
  arXiv  Detail & Related papers  (2024-03-29T14:55:40Z)
• Scaling Instructable Agents Across Many Simulated Worlds [70.97268311053328]
  Our goal is to develop an agent that can accomplish anything a human can do in any simulated 3D environment. Our approach focuses on language-driven generality while imposing minimal assumptions. Our agents interact with environments in real-time using a generic, human-like interface.
  arXiv  Detail & Related papers  (2024-03-13T17:50:32Z)
• Video as the New Language for Real-World Decision Making [100.68643056416394]
  Video data captures important information about the physical world that is difficult to express in language. Video can serve as a unified interface that can absorb internet knowledge and represent diverse tasks. We identify major impact opportunities in domains such as robotics, self-driving, and science.
  arXiv  Detail & Related papers  (2024-02-27T02:05:29Z)
• Learning to Model the World with Language [100.76069091703505]
  To interact with humans and act in the world, agents need to understand the range of language that people use and relate it to the visual world. Our key idea is that agents should interpret such diverse language as a signal that helps them predict the future. We instantiate this in Dynalang, an agent that learns a multimodal world model to predict future text and image representations.
  arXiv  Detail & Related papers  (2023-07-31T17:57:49Z)
• Build-a-Bot: Teaching Conversational AI Using a Transformer-Based Intent Recognition and Question Answering Architecture [15.19996462016215]
  This paper proposes an interface for students to learn the principles of artificial intelligence by using a natural language pipeline to train a customized model to answer questions based on their own school curriculums. The pipeline teaches students data collection, data augmentation, intent recognition, and question answering by having them work through each of these processes while creating their AI agent.
  arXiv  Detail & Related papers  (2022-12-14T22:57:44Z)
• Artificial Intelligence for the Metaverse: A Survey [66.57225253532748]
  We first deliver a preliminary of AI, including machine learning algorithms and deep learning architectures, and its role in the metaverse. We then convey a comprehensive investigation of AI-based methods concerning six technical aspects that have potentials for the metaverse. Several AI-aided applications, such as healthcare, manufacturing, smart cities, and gaming, are studied to be deployed in the virtual worlds.
  arXiv  Detail & Related papers  (2022-02-15T03:34:56Z)
• Neural Approaches to Conversational Information Retrieval [94.77863916314979]
  A conversational information retrieval (CIR) system is an information retrieval (IR) system with a conversational interface. Recent progress in deep learning has brought tremendous improvements in natural language processing (NLP) and conversational AI. This book surveys recent advances in CIR, focusing on neural approaches that have been developed in the last few years.
  arXiv  Detail & Related papers  (2022-01-13T19:04:59Z)
• GenNI: Human-AI Collaboration for Data-Backed Text Generation [102.08127062293111]
  Table2Text systems generate textual output based on structured data utilizing machine learning. GenNI (Generation Negotiation Interface) is an interactive visual system for high-level human-AI collaboration in producing descriptive text.
  arXiv  Detail & Related papers  (2021-10-19T18:07:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.