WhiteTesseract: Reframing the Interpretation of Cultural Heritage through XR and Conversational AI

• URL: http://arxiv.org/abs/2605.16972v1
• Date: Sat, 16 May 2026 12:50:37 GMT
• Title: WhiteTesseract: Reframing the Interpretation of Cultural Heritage through XR and Conversational AI
• Abstract summary: WhiteTesseract enables in-situ interpretation through high-resolution XR and conversational AI.<n>We deployed the system in a Claude Monet exhibition and conducted a controlled user study with 26 participants.<n>Results showed that WhiteTesseract significantly increased average viewing duration from 35.3 to 98.3 seconds.
• Score: 17.00861279826617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cultural heritage exhibitions often struggle to sustain attention and support reflective engagement. Physical exhibitions rely on fixed interpretive aids that lack adaptability to individual backgrounds or curiosity, and their effectiveness depends heavily on a visitor's Personal Context, prior knowledge, and cultural literacy. Meanwhile, digital exhibitions prioritize convenience and accessibility but risk weakening the Physical and Social Contexts that define embodied cultural experience. WhiteTesseract addresses this gap by enabling in-situ interpretation through high-resolution XR and conversational AI. The system integrates spatial intelligence via artwork recognition to allow visitors to selectively reduce environmental distractions (via diminished reality) and engage in context-aware dialogue (via large language models). The goal is to preserve the richness of the physical and social environment while providing a flexible space for personal reflection, enhancing Personal Context without compromising physical authenticity. We deployed the system in a Claude Monet exhibition and conducted a controlled user study with 26 participants. Quantitative results showed that WhiteTesseract modulation significantly increased average viewing duration from 35.3 to 98.3 seconds (p < 0.001). Analysis of 529 visitor-AI interactions revealed that 60% extended beyond factual queries to include analytical, emotional, and comparative inquiries. These findings demonstrate how XR and AI can enrich the physical exhibition experience by supporting deeper, more personalized engagement without displacing the embodied value of cultural heritage. We discuss technical and social constraints for real-world deployment and limitations of our controlled setting.

Related papers

• ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models [36.2474208478776]
  We introduce culture-conditioned visual emotion understanding, a task that predicts the culture-specific emotional perception of a given image.<n>We present ArtECulture, a benchmark containing 6,792 artworks with culture-specific emotion labels and explanations across English, Chinese, and Arabic cultures.
  arXiv  Detail & Related papers  (2026-08-04T09:05:41Z)
• CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs [59.09971492475217]
  We introduce CultureForest, a benchmark for textitCultural Norm Grounded Reasoning<n>CultureForest comprises 5,378 examples across 8 domains and 53 countries/regions.<n>Extensive experiments reveal that even top-tier models degrade substantially in open-ended settings.
  arXiv  Detail & Related papers  (2026-06-01T08:25:12Z)
• Vision Language Models are Confused Tourists [31.85723694463742]
  We introduce ConfusedTourist, a novel cultural adversarial robustness suite designed to assess Vision-Language Models (VLMs)<n>Our experiments reveal a critical vulnerability, where accuracy drops heavily under simple image-stacking perturbations and even worsens with its image-generation-based variant.<n>These findings highlight a critical challenge: visual cultural concept mixing can substantially impair even state-of-the-art VLMs.
  arXiv  Detail & Related papers  (2025-11-21T07:14:46Z)
• AI-Agents for Culturally Diverse Online Higher Education Environments [3.479757825266597]
  This chapter reviews existing research and suggests the usage of culturally aware AI-Agents to foster engagement and improve learning outcomes in culturally diverse online higher education environments.
  arXiv  Detail & Related papers  (2025-10-12T09:42:09Z)
• 'Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMs [0.0]
  This paper argues for a shift towards integrating qualitative approaches into AI alignment practices.<n> Drawing inspiration from Clifford Geertz's concept of "thick description," we propose that AI systems must produce outputs that reflect deeper cultural meanings.
  arXiv  Detail & Related papers  (2025-09-30T12:22:53Z)
• CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs [57.653830744706305]
  CultureScope is the most comprehensive evaluation framework to date for assessing cultural understanding in large language models.<n>Inspired by the cultural iceberg theory, we design a novel dimensional schema for cultural knowledge classification.<n> Experimental results demonstrate that our method can effectively evaluate cultural understanding.
  arXiv  Detail & Related papers  (2025-09-19T17:47:48Z)
• Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation [2.0467354053171243]
  We present the first comprehensive evaluation of Vision-Language Models (VLMs) cultural competence through multimodal story generation.<n>Our analysis reveals significant cultural adaptation capabilities, with rich culturally-specific vocabulary spanning names, familial terms, and geographic markers.<n>We uncover concerning limitations: cultural competence varies dramatically across architectures, some models exhibit inverse cultural alignment, and automated metrics show architectural bias contradicting human assessments.
  arXiv  Detail & Related papers  (2025-08-22T19:39:02Z)
• CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluation [61.130639734982395]
  We introduce CAIRe, a novel evaluation metric that assesses the degree of cultural relevance of an image.<n>Our framework grounds entities and concepts in the image to a knowledge base and uses factual information to give independent graded judgments for each culture label.
  arXiv  Detail & Related papers  (2025-06-10T17:16:23Z)
• A Reference Architecture for Gamified Cultural Heritage Applications Leveraging Generative AI and Augmented Reality [4.228905912230226]
  This paper presents a reference architecture for gamified cultural heritage applications leveraging generative AI and augmented reality.<n>Gamification enhances motivation, artificial intelligence enables adaptive storytelling and personalized content, and augmented reality fosters immersive, location-aware experiences.
  arXiv  Detail & Related papers  (2025-06-04T15:49:05Z)
• RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding [79.44246283490665]
  We introduce RAVENEA, a new benchmark designed to advance visual culture understanding through retrieval.<n>RAVENEA focuses on two tasks: culture-focused visual question answering (cVQA) and culture-informed image captioning (cIC)<n>We train and evaluate seven multimodal retrievers for each image query, and measure the downstream impact of retrieval-augmented inputs across fourteen state-of-the-art vision-language models.
  arXiv  Detail & Related papers  (2025-05-20T14:57:16Z)
• CAReDiO: Cultural Alignment of LLM via Representativeness and Distinctiveness Guided Data Optimization [50.90288681622152]
  Large Language Models (LLMs) more deeply integrate into human life across various regions.<n>Existing approaches develop culturally aligned LLMs through fine-tuning with culture-specific corpora.<n>We introduce CAReDiO, a novel cultural data construction framework.
  arXiv  Detail & Related papers  (2025-04-09T13:40:13Z)
• Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures [67.89565097372751]
  Gestures are an integral part of non-verbal communication, with meanings that vary across cultures.<n>As AI systems become more integrated into global applications, ensuring they do not inadvertently perpetuate cultural offenses is critical.<n>We introduce Multi-Cultural Set of Inappropriate Gestures and Nonverbal Signs (MC-SIGNS), a dataset of 288 gesture-country pairs annotated for offensiveness, cultural significance, and contextual factors.
  arXiv  Detail & Related papers  (2025-02-24T23:10:08Z)
• Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking [48.21982147529661]
  This paper introduces a novel approach for massively multicultural knowledge acquisition. Our method strategically navigates from densely informative Wikipedia documents on cultural topics to an extensive network of linked pages. Our work marks an important step towards deeper understanding and bridging the gaps of cultural disparities in AI.
  arXiv  Detail & Related papers  (2024-02-14T18:16:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.