CausalDialogue: Modeling Utterance-level Causality in Conversations
- URL: http://arxiv.org/abs/2212.10515v2
- Date: Sat, 8 Jul 2023 21:59:43 GMT
- Title: CausalDialogue: Modeling Utterance-level Causality in Conversations
- Authors: Yi-Lin Tuan, Alon Albalak, Wenda Xu, Michael Saxon, Connor Pryor, Lise
Getoor, William Yang Wang
- Abstract summary: We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing.
This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure.
We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
- Score: 83.03604651485327
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Despite their widespread adoption, neural conversation models have yet to
exhibit natural chat capabilities with humans. In this research, we examine
user utterances as causes and generated responses as effects, recognizing that
changes in a cause should produce a different effect. To further explore this
concept, we have compiled and expanded upon a new dataset called CausalDialogue
through crowd-sourcing. This dataset includes multiple cause-effect pairs
within a directed acyclic graph (DAG) structure. Our analysis reveals that
traditional loss functions struggle to effectively incorporate the DAG
structure, leading us to propose a causality-enhanced method called Exponential
Maximum Average Treatment Effect (ExMATE) to enhance the impact of causality at
the utterance level in training neural conversation models. To evaluate the
needs of considering causality in dialogue generation, we built a comprehensive
benchmark on CausalDialogue dataset using different models, inference, and
training methods. Through experiments, we find that a causality-inspired loss
like ExMATE can improve the diversity and agility of conventional loss function
and there is still room for improvement to reach human-level quality on this
new dataset.
Related papers
- DAG-aware Transformer for Causal Effect Estimation [0.8192907805418583]
Causal inference is a critical task across fields such as healthcare, economics, and the social sciences.
In this paper, we present a novel transformer-based method for causal inference that overcomes these challenges.
The core innovation of our model lies in its integration of causal Directed Acyclic Graphs (DAGs) directly into the attention mechanism.
arXiv Detail & Related papers (2024-10-13T23:17:58Z) - Most Influential Subset Selection: Challenges, Promises, and Beyond [9.479235005673683]
We study the Most Influential Subset Selection (MISS) problem, which aims to identify a subset of training samples with the greatest collective influence.
We conduct a comprehensive analysis of the prevailing approaches in MISS, elucidating their strengths and weaknesses.
We demonstrate that an adaptive version of theses which applies them iteratively, can effectively capture the interactions among samples.
arXiv Detail & Related papers (2024-09-25T20:00:23Z) - Position Debiasing Fine-Tuning for Causal Perception in Long-Term Dialogue [26.171522810457486]
The core of the dialogue system is to generate relevant, informative, and human-like responses based on extensive dialogue history.
There is a natural deficiency for such models, that is, inherent position bias.
We propose a novel method, named Causal Perception long-term Dialogue framework (CPD), which employs perturbation-based causal variable discovery method.
arXiv Detail & Related papers (2024-06-04T06:33:13Z) - Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models [16.328341121232484]
We apply causal effect estimation strategies to measure the effect of context interventions.
We investigate robustness to irrelevant changes and sensitivity to impactful changes of Transformers.
arXiv Detail & Related papers (2024-04-03T10:22:35Z) - Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic [51.967603572656266]
We introduce a consistent and theoretically grounded approach to annotating decompositional entailment.
We find that our new dataset, RDTE, has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets.
We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality.
arXiv Detail & Related papers (2024-02-22T18:55:17Z) - Sensitivity, Performance, Robustness: Deconstructing the Effect of
Sociodemographic Prompting [64.80538055623842]
sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give.
We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
arXiv Detail & Related papers (2023-09-13T15:42:06Z) - Re-mine, Learn and Reason: Exploring the Cross-modal Semantic
Correlations for Language-guided HOI detection [57.13665112065285]
Human-Object Interaction (HOI) detection is a challenging computer vision task.
We present a framework that enhances HOI detection by incorporating structured text knowledge.
arXiv Detail & Related papers (2023-07-25T14:20:52Z) - Context De-confounded Emotion Recognition [12.037240778629346]
Context-Aware Emotion Recognition (CAER) aims to perceive the emotional states of the target person with contextual information.
A long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states.
This paper provides a causality-based perspective to disentangle the models from the impact of such bias, and formulate the causalities among variables in the CAER task.
arXiv Detail & Related papers (2023-03-21T15:12:20Z) - Learning Neural Causal Models with Active Interventions [83.44636110899742]
We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process.
Our method significantly reduces the required number of interactions compared with random intervention targeting.
We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
arXiv Detail & Related papers (2021-09-06T13:10:37Z) - Efficient Causal Inference from Combined Observational and
Interventional Data through Causal Reductions [68.6505592770171]
Unobserved confounding is one of the main challenges when estimating causal effects.
We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders.
We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
arXiv Detail & Related papers (2021-03-08T14:29:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.