Characterizing Language Use in a Collaborative Situated Game

• URL: http://arxiv.org/abs/2512.03381v2
• Date: Fri, 05 Dec 2025 01:27:14 GMT
• Title: Characterizing Language Use in a Collaborative Situated Game
• Authors: Nicholas Tomlin, Naitian Zhou, Eve Fleisig, Liangyuan Chen, Téa Wright, Lauren Vinh, Laura X. Ma, Seun Eisape, Ellie French, Tingting Du, Tianjiao Zhang, Alexander Koller, Alane Suhr,
• Abstract summary: We collect a corpus of 11.5 hours of spoken human dialogue in the co-op mode of the popular Portal 2 virtual puzzle game.<n>We analyze player language and behavior, identifying a number of linguistic phenomena that rarely appear in most existing chitchat or task-oriented dialogue corpora.
• Score: 47.38055058236005
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cooperative video games, where multiple participants must coordinate by communicating and reasoning under uncertainty in complex environments, yield a rich source of language data. We collect the Portal Dialogue Corpus: a corpus of 11.5 hours of spoken human dialogue in the co-op mode of the popular Portal 2 virtual puzzle game, comprising 24.5K total utterances. We analyze player language and behavior, identifying a number of linguistic phenomena that rarely appear in most existing chitchat or task-oriented dialogue corpora, including complex spatial reference, clarification and repair, and ad-hoc convention formation. To support future analyses of language use in complex, situated, collaborative problem-solving scenarios, we publicly release the corpus, which comprises player videos, audio, transcripts, game state data, and both manual and automatic annotations of language data.

Related papers

• MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games [70.37904949359938]
  We evaluate language models in multi-turn interactions using a suite of collaborative games that require effective communication about private information.<n>We find that language models are unable to use interactive collaboration to improve over the non-interactive baseline scenario.<n>We analyze the linguistic features of these dialogues, assessing the roles of sycophancy, information density, and discourse coherence.
  arXiv  Detail & Related papers  (2026-02-27T17:13:20Z)
• Corpus of Cross-lingual Dialogues with Minutes and Detection of Misunderstandings [0.45498315114762483]
  We present a corpus of cross-lingual dialogues facilitated by automatic simultaneous speech translation.<n>The corpus consists of 5 hours of speech recordings with ASR and gold transcripts in 12 original languages and automatic and corrected translations into English.<n>For an overview of this task and its complexity, we attempt to quantify misunderstandings in cross-lingual meetings.
  arXiv  Detail & Related papers  (2025-12-23T09:56:23Z)
• KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpus [69.46707346122113]
  We propose a novel task and create a human-to-human video-driven multilingual mixed-type dialogue corpus.<n>The KwaiChat corpus contains a total of 93,209 videos and 246,080 dialogues, across 4 dialogue types, 30 domains, 4 languages, and 13 topics.<n>An analysis of 7 distinct LLMs on KwaiChat reveals that GPT-4o achieves the best performance but still cannot perform well in this situation.
  arXiv  Detail & Related papers  (2025-03-10T04:05:38Z)
• ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus [8.96693684560691]
  ZAEBUC-Spoken is a multilingual multidialectal Arabic-English speech corpus. The corpus presents a challenging set for automatic speech recognition (ASR) We take inspiration from established sets of transcription guidelines to present a set of guidelines handling issues of conversational speech, code-switching and orthography of both languages.
  arXiv  Detail & Related papers  (2024-03-27T01:19:23Z)
• ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation [79.66359274050885]
  We present ComSL, a speech-language model built atop a composite architecture of public pretrained speech-only and language-only models. Our approach has demonstrated effectiveness in end-to-end speech-to-text translation tasks.
  arXiv  Detail & Related papers  (2023-05-24T07:42:15Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)
• Emergent Communication of Generalizations [13.14792537601313]
  We argue that communicating about a single object in a shared visual context is prone to overfitting and does not encourage language useful beyond concrete reference. We propose games that require communicating generalizations over sets of objects representing abstract visual concepts. We find that these games greatly improve systematicity and interpretability of the learned languages.
  arXiv  Detail & Related papers  (2021-06-04T19:02:18Z)
• GupShup: An Annotated Corpus for Abstractive Summarization of Open-Domain Code-Switched Conversations [28.693328393260906]
  We introduce abstractive summarization of Hindi-English code-switched conversations and develop the first code-switched conversation summarization dataset. GupShup contains over 6,831 conversations in Hindi-English and their corresponding human-annotated summaries in English and Hindi-English. We train state-of-the-art abstractive summarization models and report their performances using both automated metrics and human evaluation.
  arXiv  Detail & Related papers  (2021-04-17T15:42:01Z)
• RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling [35.75880078666584]
  RiSAWOZ is a large-scale multi-domain Chinese Wizard-of-Oz dataset with Rich Semantic s. It contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues, with more than 150K utterances spanning over 12 domains.
  arXiv  Detail & Related papers  (2020-10-17T08:18:59Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.