Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation

• URL: http://arxiv.org/abs/2510.09390v1
• Date: Fri, 10 Oct 2025 13:44:40 GMT
• Title: Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation
• Authors: Mert İnan, Anthony Sicilia, Alex Xie, Saujas Vaduguru, Daniel Fried, Malihe Alikhani,
• Abstract summary: We develop a taxonomy of types of ambiguity that arise in this task and propose metrics to quantify them.<n>Our work also explores how multi-turn dialogue can reduce ambiguity, therefore, improve code accuracy by better matching user goals.
• Score: 48.63200319578052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Establishing shared goals is a fundamental step in human-AI communication. However, ambiguities can lead to outputs that seem correct but fail to reflect the speaker's intent. In this paper, we explore this issue with a focus on the data visualization domain, where ambiguities in natural language impact the generation of code that visualizes data. The availability of multiple views on the contextual (e.g., the intended plot and the code rendering the plot) allows for a unique and comprehensive analysis of diverse ambiguity types. We develop a taxonomy of types of ambiguity that arise in this task and propose metrics to quantify them. Using Matplotlib problems from the DS-1000 dataset, we demonstrate that our ambiguity metrics better correlate with human annotations than uncertainty baselines. Our work also explores how multi-turn dialogue can reduce ambiguity, therefore, improve code accuracy by better matching user goals. We evaluate three pragmatic models to inform our dialogue strategies: Gricean Cooperativity, Discourse Representation Theory, and Questions under Discussion. A simulated user study reveals how pragmatic dialogues reduce ambiguity and enhance code accuracy, highlighting the value of multi-turn exchanges in code generation.

Related papers

• Trick or Neat: Adversarial Ambiguity and Language Model Evaluation [42.62991342963119]
  We assess language models' sensitivity to ambiguity by introducing an adversarial ambiguity dataset.<n>Our findings show that direct prompting fails to robustly identify ambiguity, while linear probes trained on model representations can decode ambiguity with high accuracy.
  arXiv  Detail & Related papers  (2025-06-01T22:50:06Z)
• VAGUE: Visual Contexts Clarify Ambiguous Expressions [26.190504250419547]
  VAGUE is a benchmark evaluating multimodal AI systems' ability to integrate visual context for intent.<n>Our experiments reveal that existing multimodal AI models struggle to infer the speaker's true intent.<n>Analysis of failure cases demonstrates that current models fail to distinguish true intent from superficial correlations in the visual scene.
  arXiv  Detail & Related papers  (2024-11-21T14:01:42Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• We're Afraid Language Models Aren't Modeling Ambiguity [136.8068419824318]
  Managing ambiguity is a key part of human language understanding. We characterize ambiguity in a sentence by its effect on entailment relations with another sentence. We show that a multilabel NLI model can flag political claims in the wild that are misleading due to ambiguity.
  arXiv  Detail & Related papers  (2023-04-27T17:57:58Z)
• Is the Elephant Flying? Resolving Ambiguities in Text-to-Image Generative Models [64.58271886337826]
  We study ambiguities that arise in text-to-image generative models. We propose a framework to mitigate ambiguities in the prompts given to the systems by soliciting clarifications from the user.
  arXiv  Detail & Related papers  (2022-11-17T17:12:43Z)
• Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning [2.5567566997688043]
  We introduce Curved Contrastive Learning (CCL), a novel representation learning technique for learning the relative turn distance between utterance pairs in multi-turn dialogues. The resulting bi-encoder models can guide transformers as a response ranking model towards a goal in a zero-shot fashion by projecting the goal utterance and the corresponding reply candidates into a latent space.
  arXiv  Detail & Related papers  (2022-11-14T18:16:48Z)
• Contextual information integration for stance detection via cross-attention [59.662413798388485]
  Stance detection deals with identifying an author's stance towards a target. Most existing stance detection models are limited because they do not consider relevant contextual information. We propose an approach to integrate contextual information as text.
  arXiv  Detail & Related papers  (2022-11-03T15:04:29Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.