Evaluating Usability and Engagement of Large Language Models in Virtual Reality for Traditional Scottish Curling

• URL: http://arxiv.org/abs/2408.09285v2
• Date: Wed, 25 Sep 2024 13:06:53 GMT
• Title: Evaluating Usability and Engagement of Large Language Models in Virtual Reality for Traditional Scottish Curling
• Authors: Ka Hei Carrie Lau, Efe Bozkir, Hong Gao, Enkelejda Kasneci,
• Abstract summary: This paper explores the innovative application of Large Language Models (LLMs) in Virtual Reality (VR) environments. It focuses on traditional Scottish curling presented in the game Scottish Bonspiel VR''
• Score: 9.91445427832401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores the innovative application of Large Language Models (LLMs) in Virtual Reality (VR) environments to promote heritage education, focusing on traditional Scottish curling presented in the game ``Scottish Bonspiel VR''. Our study compares the effectiveness of LLM-based chatbots with pre-defined scripted chatbots, evaluating key criteria such as usability, user engagement, and learning outcomes. The results show that LLM-based chatbots significantly improve interactivity and engagement, creating a more dynamic and immersive learning environment. This integration helps document and preserve cultural heritage and enhances dissemination processes, which are crucial for safeguarding intangible cultural heritage (ICH) amid environmental changes. Furthermore, the study highlights the potential of novel technologies in education to provide immersive experiences that foster a deeper appreciation of cultural heritage. These findings support the wider application of LLMs and VR in cultural education to address global challenges and promote sustainable practices to preserve and enhance cultural heritage.

Related papers

• From Word to World: Evaluate and Mitigate Culture Bias via Word Association Test [48.623761108859085]
  We extend the human-centered word association test (WAT) to assess the alignment of large language models with cross-cultural cognition.<n>To mitigate the culture preference, we propose CultureSteer, an innovative approach that integrates a culture-aware steering mechanism.
  arXiv  Detail & Related papers  (2025-05-24T07:05:10Z)
• From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs [57.43233760384488]
  Adapting cultural values in Large Language Models (LLMs) presents significant challenges.<n>Prior work primarily aligns LLMs with different cultural values using World Values Survey (WVS) data.<n>In this paper, we investigate WVS-based training for cultural value adaptation and find that relying solely on survey data cane cultural norms and interfere with factual knowledge.
  arXiv  Detail & Related papers  (2025-05-22T09:00:01Z)
• Cultural Learning-Based Culture Adaptation of Language Models [70.1063219524999]
  Adapting large language models (LLMs) to diverse cultural values is a challenging task. We present CLCA, a novel framework for enhancing LLM alignment with cultural values based on cultural learning.
  arXiv  Detail & Related papers  (2025-04-03T18:16:26Z)
• CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries [63.00147630084146]
  Vision-language models (VLMs) have advanced human-AI interaction but struggle with cultural understanding. CultureVerse is a large-scale multimodal benchmark covering 19, 682 cultural concepts, 188 countries/regions, 15 cultural concepts, and 3 question types. We propose CultureVLM, a series of VLMs fine-tuned on our dataset to achieve significant performance improvement in cultural understanding.
  arXiv  Detail & Related papers  (2025-01-02T14:42:37Z)
• LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Models [62.47865866398233]
  This white paper proposes a framework to generate linguistic tools for low-resource languages. By addressing the data scarcity that hinders intelligent applications for such languages, we contribute to promoting linguistic diversity.
  arXiv  Detail & Related papers  (2024-11-20T16:59:41Z)
• Survey of Cultural Awareness in Language Models: Text and Beyond [39.77033652289063]
  Large-scale deployment of large language models (LLMs) in various applications requires LLMs to be culturally sensitive to the user to ensure inclusivity. Culture has been widely studied in psychology and anthropology, and there has been a recent surge in research on making LLMs more culturally inclusive.
  arXiv  Detail & Related papers  (2024-10-30T16:37:50Z)
• Leveraging Retrieval-Augmented Generation for Culturally Inclusive Hakka Chatbots: Design Insights and User Perceptions [4.388667614435888]
  This study introduces a groundbreaking approach to promoting and safeguarding the rich heritage of Taiwanese Hakka culture. By integrating external databases with generative AI models, RAG technology bridges this gap. This is particularly significant in an age where digital platforms often dilute cultural identities.
  arXiv  Detail & Related papers  (2024-10-21T01:36:08Z)
• Investigating the Role of Cultural Values in Adopting Large Language Models for Software Engineering [17.818350887316004]
  This study focuses on the role of professionals' cultural values in the adoption of Large Language Models (LLMs) in software development. We found that habit and performance expectancy are the primary drivers of LLM adoption, while cultural values do not significantly moderate this process.
  arXiv  Detail & Related papers  (2024-09-08T10:58:45Z)
• Benchmarking Cognitive Domains for LLMs: Insights from Taiwanese Hakka Culture [4.467334566487944]
  This study introduces a benchmark designed to evaluate the performance of large language models (LLMs) in understanding and processing cultural knowledge. The study develops a multi-dimensional framework that systematically assesses LLMs across six cognitive domains: Remembering, Understanding, Applying, Analyzing, evaluating, and Creating. The results highlight the effectiveness of RAG in improving accuracy across all cognitive domains, particularly in tasks requiring precise retrieval and application of cultural knowledge.
  arXiv  Detail & Related papers  (2024-09-03T02:50:04Z)
• Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense [98.09670425244462]
  Large language models (LLMs) have demonstrated substantial commonsense understanding. This paper examines the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks.
  arXiv  Detail & Related papers  (2024-05-07T20:28:34Z)
• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge [69.82940934994333]
  We introduce CulturalTeaming, an interactive red-teaming system that leverages human-AI collaboration to build challenging evaluation dataset. Our study reveals that CulturalTeaming's various modes of AI assistance support annotators in creating cultural questions. CULTURALBENCH-V0.1 is a compact yet high-quality evaluation dataset with users' red-teaming attempts.
  arXiv  Detail & Related papers  (2024-04-10T00:25:09Z)
• Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking [48.21982147529661]
  This paper introduces a novel approach for massively multicultural knowledge acquisition. Our method strategically navigates from densely informative Wikipedia documents on cultural topics to an extensive network of linked pages. Our work marks an important step towards deeper understanding and bridging the gaps of cultural disparities in AI.
  arXiv  Detail & Related papers  (2024-02-14T18:16:54Z)
• Rethinking Machine Unlearning for Large Language Models [85.92660644100582]
  We explore machine unlearning in the domain of large language models (LLMs) This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities.
  arXiv  Detail & Related papers  (2024-02-13T20:51:58Z)
• CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models [41.885600036131045]
  CDEval is a benchmark aimed at evaluating the cultural dimensions of Large Language Models. It is constructed by incorporating both GPT-4's automated generation and human verification, covering six cultural dimensions across seven domains.
  arXiv  Detail & Related papers  (2023-11-28T02:01:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.