Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA

• URL: http://arxiv.org/abs/2211.07516v2
• Date: Thu, 1 Jun 2023 19:19:23 GMT
• Title: Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA
• Authors: Elias Stengel-Eskin, Jimena Guallar-Blasco, Yi Zhou, Benjamin Van Durme
• Abstract summary: Resolving ambiguous questions is key to successfully answering them. We create a dataset of ambiguous examples, grouping answers by the underlying question they address and rephrasing the question for each group to reduce ambiguity. We then develop an English question-generation model which we demonstrate via automatic and human evaluation produces less ambiguous questions.
• Score: 33.11688014628816
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural language is ambiguous. Resolving ambiguous questions is key to successfully answering them. Focusing on questions about images, we create a dataset of ambiguous examples. We annotate these, grouping answers by the underlying question they address and rephrasing the question for each group to reduce ambiguity. Our analysis reveals a linguistically-aligned ontology of reasons for ambiguity in visual questions. We then develop an English question-generation model which we demonstrate via automatic and human evaluation produces less ambiguous questions. We further show that the question generation objective we use allows the model to integrate answer group information without any direct supervision.

Related papers

• Open Domain Question Answering with Conflicting Contexts [55.739842087655774]
  We find that as much as 25% of unambiguous, open domain questions can lead to conflicting contexts when retrieved using Google Search. We ask our annotators to provide explanations for their selections of correct answers.
  arXiv  Detail & Related papers  (2024-10-16T07:24:28Z)
• Detecting Temporal Ambiguity in Questions [16.434748534272014]
  Temporally ambiguous questions are one of the most common types of such questions. Our annotations focus on capturing temporal ambiguity to study the task of detecting temporally ambiguous questions. We propose a novel approach by using diverse search strategies based on disambiguated versions of the questions.
  arXiv  Detail & Related papers  (2024-09-25T15:59:58Z)
• Which questions should I answer? Salience Prediction of Inquisitive Questions [118.097974193544]
  We show that highly salient questions are empirically more likely to be answered in the same article. We further validate our findings by showing that answering salient questions is an indicator of summarization quality in news.
  arXiv  Detail & Related papers  (2024-04-16T21:33:05Z)
• Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations [70.6395572287422]
  Self-alignment method is capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. We conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired.
  arXiv  Detail & Related papers  (2024-02-23T02:24:36Z)
• Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions [95.92276099234344]
  We present a new state-of-the-art for answering ambiguous questions that exploits a database of unambiguous questions generated from Wikipedia. Our method improves performance by 15% on recall measures and 10% on measures which evaluate disambiguating questions from predicted outputs.
  arXiv  Detail & Related papers  (2023-08-16T20:23:16Z)
• Weakly Supervised Visual Question Answer Generation [2.7605547688813172]
  We present a weakly supervised method that synthetically generates question-answer pairs procedurally from visual information and captions. We perform an exhaustive experimental analysis on VQA dataset and see that our model significantly outperforms SOTA methods on BLEU scores.
  arXiv  Detail & Related papers  (2023-06-11T08:46:42Z)
• Selectively Answering Ambiguous Questions [38.83930394700588]
  We find that the most reliable approach to decide when to abstain involves quantifying repetition within sampled model outputs. Our results suggest that sampling-based confidence scores help calibrate answers to relatively unambiguous questions.
  arXiv  Detail & Related papers  (2023-05-24T01:25:38Z)
• WikiWhy: Answering and Explaining Cause-and-Effect Questions [62.60993594814305]
  We introduce WikiWhy, a QA dataset built around explaining why an answer is true in natural language. WikiWhy contains over 9,000 "why" question-answer-rationale triples, grounded on Wikipedia facts across a diverse set of topics. GPT-3 baselines achieve only 38.7% human-evaluated correctness in the end-to-end answer & explain condition.
  arXiv  Detail & Related papers  (2022-10-21T17:59:03Z)
• Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion [57.43781399856913]
  This work adopts the linguistic framework of Questions Under Discussion (QUD) for discourse analysis. We characterize relationships between sentences as free-form questions, in contrast to exhaustive fine-grained questions. We develop the first-of-its-kind QUD that derives a dependency structure of questions over full documents.
  arXiv  Detail & Related papers  (2022-10-12T03:53:12Z)
• Open-domain clarification question generation without question examples [4.34222556313791]
  We propose a framework for building a question-asking model capable of producing polar (yes-no) clarification questions. Our model uses an expected information gain objective to derive informative questions from an off-the-shelf image captioner. We demonstrate our model's ability to pose questions that improve communicative success in a goal-oriented 20 questions game with synthetic and human answerers.
  arXiv  Detail & Related papers  (2021-10-19T07:51:54Z)
• Rephrasing visual questions by specifying the entropy of the answer distribution [0.0]
  We propose a novel task, rephrasing the questions by controlling the ambiguity of the questions. The ambiguity of a visual question is defined by the use of the entropy of the answer distribution predicted by a VQA model. We demonstrate the advantage of our approach that can control the ambiguity of the rephrased questions, and an interesting observation that it is harder to increase than to reduce ambiguity.
  arXiv  Detail & Related papers  (2020-04-10T09:32:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.