"I'd rather just go to bed": Understanding Indirect Answers

• URL: http://arxiv.org/abs/2010.03450v1
• Date: Wed, 7 Oct 2020 14:41:40 GMT
• Title: "I'd rather just go to bed": Understanding Indirect Answers
• Authors: Annie Louis, Dan Roth, and Filip Radlinski
• Abstract summary: We revisit a pragmatic inference problem in dialog: understanding indirect responses to questions. We create and release the first large-scale English language corpus 'Circa' with 34,268 (polar question, indirect answer) pairs. We present BERT-based neural models to predict such categories for a question-answer pair.
• Score: 61.234722570671686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We revisit a pragmatic inference problem in dialog: understanding indirect responses to questions. Humans can interpret 'I'm starving.' in response to 'Hungry?', even without direct cue words such as 'yes' and 'no'. In dialog systems, allowing natural responses rather than closed vocabularies would be similarly beneficial. However, today's systems are only as sensitive to these pragmatic moves as their language model allows. We create and release the first large-scale English language corpus 'Circa' with 34,268 (polar question, indirect answer) pairs to enable progress on this task. The data was collected via elaborate crowdsourcing, and contains utterances with yes/no meaning, as well as uncertain, middle-ground, and conditional responses. We also present BERT-based neural models to predict such categories for a question-answer pair. We find that while transfer learning from entailment works reasonably, performance is not yet sufficient for robust dialog. Our models reach 82-88% accuracy for a 4-class distinction, and 74-85% for 6 classes.

Related papers

• Quirk or Palmer: A Comparative Study of Modal Verb Frameworks with Annotated Datasets [8.09076910034882]
  Linguists have yet to agree on a single framework for the categorization of modal verb senses. This work presents Moverb dataset, which consists of 27,240 annotations of modal verb senses over 4,540 utterances.
  arXiv  Detail & Related papers  (2022-12-20T10:44:18Z)
• Prompting for a conversation: How to control a dialog model? [9.268682116424518]
  Dialog models are trained on a large amount of text, yet their responses need to be limited to a desired scope and style of a dialog agent. Because the datasets used to achieve the former contain language that is not compatible with the latter, pre-trained dialog models are fine-tuned on smaller curated datasets. In this paper we investigate if prompting can mitigate the above trade-off.
  arXiv  Detail & Related papers  (2022-09-22T14:59:55Z)
• Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension [48.483910831143724]
  Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances. We develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input.
  arXiv  Detail & Related papers  (2022-03-19T05:20:25Z)
• Open-domain clarification question generation without question examples [4.34222556313791]
  We propose a framework for building a question-asking model capable of producing polar (yes-no) clarification questions. Our model uses an expected information gain objective to derive informative questions from an off-the-shelf image captioner. We demonstrate our model's ability to pose questions that improve communicative success in a goal-oriented 20 questions game with synthetic and human answerers.
  arXiv  Detail & Related papers  (2021-10-19T07:51:54Z)
• Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? [87.20342701232869]
  We investigate the abilities of ungrounded systems to acquire meaning. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation if all expressions in the language are referentially transparent. However, if the language uses non-transparent patterns like variable binding, we show that emulation can become an uncomputable problem.
  arXiv  Detail & Related papers  (2021-04-22T01:00:17Z)
• My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism [71.34160809068996]
  Recent work shows that automated scoring systems are prone to even common-sense adversarial samples. We utilize recent advances in interpretability to find the extent to which features such as coherence, content and relevance are important for automated scoring mechanisms. We also find that since the models are not semantically grounded with world-knowledge and common sense, adding false facts such as the world is flat'' actually increases the score instead of decreasing it.
  arXiv  Detail & Related papers  (2020-12-27T06:19:20Z)
• A Taxonomy of Empathetic Response Intents in Human Social Conversations [1.52292571922932]
  Open-domain conversational agents are becoming increasingly popular in the natural language processing community. One of the challenges is enabling them to converse in an empathetic manner. Current neural response generation methods rely solely on end-to-end learning from large scale conversation data to generate dialogues. Recent work has shown the promise of combining dialogue act/intent modelling and neural response generation.
  arXiv  Detail & Related papers  (2020-12-07T21:56:45Z)
• Generating Dialogue Responses from a Semantic Latent Space [75.18449428414736]
  We propose an alternative to the end-to-end classification on vocabulary. We learn the pair relationship between the prompts and responses as a regression task on a latent space. Human evaluation showed that learning the task on a continuous space can generate responses that are both relevant and informative.
  arXiv  Detail & Related papers  (2020-10-04T19:06:16Z)
• MuTual: A Dataset for Multi-Turn Dialogue Reasoning [53.10434937685962]
  MuTual is a novel dataset for Multi-Turn dialogue Reasoning. It consists of 8,860 manually annotated dialogues based on Chinese student English listening comprehension exams. We show that state-of-the-art methods only reach 71%, which is far behind the human performance of 94%.
  arXiv  Detail & Related papers  (2020-04-09T11:42:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.