Modeling Cross-Cultural Pragmatic Inference with Codenames Duet

• URL: http://arxiv.org/abs/2306.02475v1
• Date: Sun, 4 Jun 2023 20:47:07 GMT
• Title: Modeling Cross-Cultural Pragmatic Inference with Codenames Duet
• Authors: Omar Shaikh, Caleb Ziems, William Held, Aryan J. Pariani, Fred Morstatter, Diyi Yang
• Abstract summary: This paper introduces the Cultural Codes dataset, which operationalizes sociocultural pragmatic inference in a simple word reference game. Our dataset consists of 794 games with 7,703 turns, distributed across 153 unique players. Our experiments show that accounting for background characteristics significantly improves model performance for tasks related to clue giving and guessing.
• Score: 40.52354928048333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pragmatic reference enables efficient interpersonal communication. Prior work uses simple reference games to test models of pragmatic reasoning, often with unidentified speakers and listeners. In practice, however, speakers' sociocultural background shapes their pragmatic assumptions. For example, readers of this paper assume NLP refers to "Natural Language Processing," and not "Neuro-linguistic Programming." This work introduces the Cultural Codes dataset, which operationalizes sociocultural pragmatic inference in a simple word reference game. Cultural Codes is based on the multi-turn collaborative two-player game, Codenames Duet. Our dataset consists of 794 games with 7,703 turns, distributed across 153 unique players. Alongside gameplay, we collect information about players' personalities, values, and demographics. Utilizing theories of communication and pragmatics, we predict each player's actions via joint modeling of their sociocultural priors and the game context. Our experiments show that accounting for background characteristics significantly improves model performance for tasks related to both clue giving and guessing, indicating that sociocultural priors play a vital role in gameplay decisions.

Related papers

• Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests [89.09172401497213]
  We examine three evaluation paradigms: large question-answering benchmarks, interactive games, and cognitive tests. We compile a suite of targeted tests that measure cognitive abilities deemed essential for effective language use. Our analyses reveal that interactive games are superior to standard benchmarks in discriminating models.
  arXiv  Detail & Related papers  (2025-02-20T08:36:58Z)
• Improving Cooperation in Language Games with Bayesian Inference and the Cognitive Hierarchy [0.8149787238021642]
  In language games, failure may be due to disagreement in the understanding of either the semantics or pragmatics of an utterance. We model coarse uncertainty in semantics using a prior distribution of language models and uncertainty in pragmatics using the cognitive hierarchy. To handle all forms of uncertainty we construct agents that learn the behavior of their partner using Bayesian inference.
  arXiv  Detail & Related papers  (2024-12-16T23:24:12Z)
• Understanding Players as if They Are Talking to the Game in a Customized Language: A Pilot Study [3.4333699338998693]
  This pilot study explores the application of language models (LMs) to model game event sequences. We transform raw event data into textual sequences and pretraining a Longformer model on this data. The results demonstrate the potential of self-supervised LMs in enhancing game design and personalization without relying on ground-truth labels.
  arXiv  Detail & Related papers  (2024-10-24T09:59:10Z)
• Communicate to Play: Pragmatic Reasoning for Efficient Cross-Cultural Communication in Codenames [0.0]
  We develop our method Rational Speech Acts for Cross-Cultural Communication (RSA+C3) to resolve cross-cultural differences in common ground. We show that our method successfully improves collaboration between simulated players of different cultures.
  arXiv  Detail & Related papers  (2024-08-09T07:02:18Z)
• Visually Grounded Language Learning: a review of language games, datasets, tasks, and models [60.2604624857992]
  Many Vision+Language (V+L) tasks have been defined with the aim of creating models that can ground symbols in the visual modality. In this work, we provide a systematic literature review of several tasks and models proposed in the V+L field.
  arXiv  Detail & Related papers  (2023-12-05T02:17:29Z)
• Modeling Player Personality Factors from In-Game Behavior and Affective Expression [17.01727448431269]
  We explore possibilities to predict a series of player personality questionnaire metrics from recorded in-game behavior. We predict a wide variety of personality metrics from seven established questionnaires across 62 players over 60 minute gameplay of a customized version of the role-playing game Fallout: New Vegas.
  arXiv  Detail & Related papers  (2023-08-27T22:59:08Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions [75.42526766746515]
  We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs. Our dataset focuses on the assessment of functional commonsense knowledge rules rather than factual knowledge. Experiments show that the introduced dataset is challenging to previous machine reading models as well as the new large language models.
  arXiv  Detail & Related papers  (2022-10-18T19:20:53Z)
• Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia [4.215251065887862]
  We analyze the effect of speaker role on language use through the game of Mafia. We show that classification models are able to rank deceptive players as more suspicious than honest ones. We present methods for using our trained models to identify features that distinguish between player roles.
  arXiv  Detail & Related papers  (2022-07-05T18:29:27Z)
• Contextual Games: Multi-Agent Learning with Side Information [57.76996806603094]
  We formulate the novel class of contextual games driven by contextual information at each round. By means of kernel-based regularity assumptions, we model the correlation between different contexts and game outcomes. We propose a novel online (meta) algorithm that exploits such correlations to minimize the contextual regret of individual players.
  arXiv  Detail & Related papers  (2021-07-13T18:37:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.