Contrastive Learning for Inference in Dialogue

• URL: http://arxiv.org/abs/2310.12467v2
• Date: Mon, 13 Nov 2023 04:18:58 GMT
• Title: Contrastive Learning for Inference in Dialogue
• Authors: Etsuko Ishii, Yan Xu, Bryan Wilie, Ziwei Ji, Holy Lovenia, Willy Chung, Pascale Fung
• Abstract summary: Inference, especially those derived from inductive processes, is a crucial component in our conversation. Recent large language models show remarkable advances in inference tasks. But their performance in inductive reasoning, where not all information is present in the context, is far behind deductive reasoning.
• Score: 56.20733835058695
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Inference, especially those derived from inductive processes, is a crucial component in our conversation to complement the information implicitly or explicitly conveyed by a speaker. While recent large language models show remarkable advances in inference tasks, their performance in inductive reasoning, where not all information is present in the context, is far behind deductive reasoning. In this paper, we analyze the behavior of the models based on the task difficulty defined by the semantic information gap -- which distinguishes inductive and deductive reasoning (Johnson-Laird, 1988, 1993). Our analysis reveals that the disparity in information between dialogue contexts and desired inferences poses a significant challenge to the inductive inference process. To mitigate this information gap, we investigate a contrastive learning approach by feeding negative samples. Our experiments suggest negative samples help models understand what is wrong and improve their inference generations.

Related papers

• Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variations [13.749495524988774]
  We systematically investigate the impact of real-life variations on state-of-the-art dialogue summarization models. We introduce two types of perturbations: utterance-level perturbations that modify individual utterances with errors and language variations, and dialogue-level perturbations that add non-informative exchanges. We find that both fine-tuned and instruction-tuned models are affected by input variations, with the latter being more susceptible.
  arXiv  Detail & Related papers  (2023-11-15T05:11:43Z)
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
  We investigate the ability to model unusual, unexpected, and unlikely situations. Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation. We release a new English language corpus called UNcommonsense.
  arXiv  Detail & Related papers  (2023-11-14T19:00:55Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Causal interventions expose implicit situation models for commonsense language understanding [3.290878132806227]
  We analyze performance on the Winograd Challenge, where a single context cue shifts interpretation of an ambiguous pronoun. We identify a circuit of attention heads that are responsible for propagating information from the context word. These analyses suggest distinct pathways through which implicit situation models are constructed to guide pronoun resolution.
  arXiv  Detail & Related papers  (2023-06-06T17:36:43Z)
• Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations [118.0818807474809]
  Abductive reasoning aims to find plausible explanations for an event. Existing approaches for abductive reasoning in natural language processing often rely on manually generated annotations for supervision. This work proposes an approach for abductive commonsense reasoning that exploits the fact that only a subset of explanations is correct for a given context.
  arXiv  Detail & Related papers  (2023-05-24T01:35:10Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [56.85319224208865]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account. We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed. Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem [15.476899850339395]
  We introduce the task of implicit offensive text detection in dialogues. We argue that reasoning is crucial for understanding this broader class of offensive utterances. We release SLIGHT, a dataset to support research on this task.
  arXiv  Detail & Related papers  (2022-04-22T06:20:15Z)
• Coreference-Aware Dialogue Summarization [24.986030179701405]
  We investigate approaches to explicitly incorporate coreference information in neural abstractive dialogue summarization models. Experimental results show that the proposed approaches achieve state-of-the-art performance. Evaluation results on factual correctness suggest such coreference-aware models are better at tracing the information flow among interlocutors.
  arXiv  Detail & Related papers  (2021-06-16T05:18:50Z)
• Towards Interpretable Reasoning over Paragraph Effects in Situation [126.65672196760345]
  We focus on the task of reasoning over paragraph effects in situation, which requires a model to understand the cause and effect. We propose a sequential approach for this task which explicitly models each step of the reasoning process with neural network modules. In particular, five reasoning modules are designed and learned in an end-to-end manner, which leads to a more interpretable model.
  arXiv  Detail & Related papers  (2020-10-03T04:03:52Z)
• Quantifying the Causal Effects of Conversational Tendencies [17.506263520769927]
  Drawing causal links between conversational behaviors and outcomes is a necessary step in using them in a prescriptive fashion. We focus on the task of determining a particular type of policy for a text-based crisis counseling platform. We show how to circumvent these inference challenges in our particular domain.
  arXiv  Detail & Related papers  (2020-09-08T18:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.