Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

• URL: http://arxiv.org/abs/2606.08081v1
• Date: Sat, 06 Jun 2026 10:05:49 GMT
• Title: Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions
• Abstract summary: We compare capable multimodal agent dyads with human dyads from the KTH Tangrams corpus.<n>Humans reduce effort through entrainment, compressing descriptions and increasing label alignment with partners.<n>Agents instead maintain fixed effort levels, producing verbose descriptions from round one, with near-ceiling label overlap that is statistically indistinguishable between real and pseudo dyads.
• Score: 2.3410384770553154
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in shared interaction history. Prior work shows that multimodal LLMs fail to become more efficient across rounds, although they align on the labels they use. How can we determine whether this alignment reflects partner-specific grounding rather than a shared task vocabulary? We address this question by comparing capable multimodal agent dyads with human dyads from the KTH Tangrams corpus. Our novel methodological contribution is a constrained pseudo-dyad baseline that matches the original referential task structure, but breaks partner history. This baseline enables us to test whether the observed label alignment depends on interaction with a specific partner. Across three analytic layers (task competence, description strategy, alignment dynamics), we find clear differences. Humans reduce effort through entrainment, compressing descriptions and increasing label alignment with partners. Agents instead maintain fixed effort levels, producing verbose descriptions from round one, with near-ceiling label overlap that is statistically indistinguishable between real and pseudo dyads. MLLMs thus achieve coordination without convention, succeeding by verbose description rather than by forming the compact, history-dependent referring expressions characteristic of human dialogue.

Related papers

• EnsembleLink: Accurate Record Linkage Without Training Data [0.0]
  I present EnsembleLink, a method that achieves high accuracy without any training labels.<n>On benchmarks spanning city names, person names, multilingual political parties organizations records, EnsembleLink matches or exceeds methods requiring extensive labeling.<n>The method runs locally on open-source models, requiring no external API calls, and completes typical linkage tasks in minutes.
  arXiv  Detail & Related papers  (2026-01-29T00:41:11Z)
• A Simple Yet Strong Baseline for Long-Term Conversational Memory of LLM Agents [4.706565675142042]
  LLM-based conversational agents still struggle to maintain coherent, personalized interaction over many sessions.<n>Motivated by neo-Davidsonian event semantics, we propose an event-centric alternative that represents conversational history as short, event-like propositions.<n>Our design aims to preserve information in a non-compressive form and make it more accessible, rather than more lossy.
  arXiv  Detail & Related papers  (2025-11-21T12:41:17Z)
• Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection [26.403716144346756]
  Few-shot Multi-label Intent Detection (MID) is crucial for dialogue systems, aiming to detect multiple intents of utterances.<n>We propose a multi-label joint learning method for few-shot MID in an end-to-end manner.<n> Experiments show that we outperform strong baselines by an average of 9.54% AUC and 11.19% Macro-F1 in 1-shot scenarios.
  arXiv  Detail & Related papers  (2025-10-09T04:47:06Z)
• QUDsim: Quantifying Discourse Similarities in LLM-Generated Text [70.22275200293964]
  We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression.<n>We then use this framework to build $textbfQUDsim$, a similarity metric that can detect discursive parallels between documents.<n>Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs.
  arXiv  Detail & Related papers  (2025-04-12T23:46:09Z)
• Multi-Label Classification for Implicit Discourse Relation Recognition [10.280148603465697]
  We explore various multi-label classification frameworks to handle implicit discourse relation recognition. We show that multi-label classification methods don't depress performance for single-label prediction.
  arXiv  Detail & Related papers  (2024-06-06T19:37:25Z)
• Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents [61.63208012250885]
  We formulate recognizing semantic differences as a token-level regression task. We study three unsupervised approaches that rely on a masked language model. Our results show that an approach based on word alignment and sentence-level contrastive learning has a robust correlation to gold labels.
  arXiv  Detail & Related papers  (2023-05-22T17:58:04Z)
• Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
  We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings. RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
  arXiv  Detail & Related papers  (2022-12-17T05:25:17Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [60.62039705180484]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text.<n> Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• R$^2$-Net: Relation of Relation Learning Network for Sentence Semantic Matching [58.72111690643359]
  We propose a Relation of Relation Learning Network (R2-Net) for sentence semantic matching. We first employ BERT to encode the input sentences from a global perspective. Then a CNN-based encoder is designed to capture keywords and phrase information from a local perspective. To fully leverage labels for better relation information extraction, we introduce a self-supervised relation of relation classification task.
  arXiv  Detail & Related papers  (2020-12-16T13:11:30Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.