Related papers: Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning

Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning

URL: http://arxiv.org/abs/2305.14970v2
Date: Mon, 8 Apr 2024 14:59:53 GMT
Title: Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning
Authors: Tianqing Fang, Zhaowei Wang, Wenxuan Zhou, Hongming Zhang, Yangqiu Song, Muhao Chen,
Abstract summary: Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. Knowledge conflicts arise when there is a mismatch between the actual temporal relations of events in the context and the prior knowledge or biases learned by the model.
Score: 87.92209048521153
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. However, knowledge conflicts arise when there is a mismatch between the actual temporal relations of events in the context and the prior knowledge or biases learned by the model. In this paper, we propose to detect knowledge-conflict examples in event temporal reasoning using bias indicators, which include event relation prior bias, tense bias, narrative bias, and dependency bias. We define conflict examples as those where event relations are opposite to biased or prior relations. To mitigate event-related knowledge conflicts, we introduce a Counterfactual Data Augmentation (CDA) based method that can be applied to both Pre-trained Language Models (PLMs) and Large Language Models (LLMs) either as additional training data or demonstrations for In-Context Learning. Experiments suggest both PLMs and LLMs suffer from knowledge conflicts in event temporal reasoning, and CDA has the potential for reducing hallucination and improving model performance.

Related papers

From Internal Conflict to Contextual Adaptation of Language Models [42.776896363518844]
Studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training. We introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature. Our experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context.
arXiv Detail & Related papers (2024-07-24T06:06:07Z)
Studying Large Language Model Behaviors Under Realistic Knowledge Conflicts [54.953320616069654]
We present a framework for studying knowledge conflicts in a realistic setup. We update incorrect parametric knowledge using real conflicting documents. In cases where the models still fail to update their answers, we find a parametric bias.
arXiv Detail & Related papers (2024-04-24T17:59:36Z)
Enhancing Event Causality Identification with Rationale and Structure-Aware Causal Question Answering [30.000134835133522]
Event Causality Identification (DECI) aims to identify causal relations between two events in documents. Recent research tends to use pre-trained language models to generate the event causal relations. We propose a multi-task learning framework to enhance event causality identification with rationale and structure-aware causal question answering.
arXiv Detail & Related papers (2024-03-17T07:41:58Z)
Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
arXiv Detail & Related papers (2022-12-20T17:40:03Z)
Logic and Commonsense-Guided Temporal Knowledge Graph Completion [9.868206060374991]
A temporal knowledge graph (TKG) stores the events derived from the data involving time. We propose a Logic and Commonsense-Guided Embedding model (LCGE) to jointly learn the time-sensitive representation involving timeliness and causality of events.
arXiv Detail & Related papers (2022-11-30T10:06:55Z)
EA$^2$E: Improving Consistency with Event Awareness for Document-Level Argument Extraction [52.43978926985928]
We introduce the Event-Aware Argument Extraction (EA$2$E) model with augmented context for training and inference. Experiment results on WIKIEVENTS and ACE2005 datasets demonstrate the effectiveness of EA$2$E.
arXiv Detail & Related papers (2022-05-30T04:33:51Z)
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification [74.6318379374801]
We propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios.
arXiv Detail & Related papers (2022-03-04T10:11:15Z)
Causal Knowledge Guided Societal Event Forecasting [24.437437565689393]
We introduce a deep learning framework that integrates causal effect estimation into event forecasting. Two robust learning modules, including a feature reweighting module and an approximate loss, are introduced to enable prior knowledge injection.
arXiv Detail & Related papers (2021-12-10T17:41:02Z)
Temporal Reasoning on Implicit Events from Distant Supervision [91.20159064951487]
We propose a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events. We find that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. We propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from large-scale text and uses temporal rules to infer end times.
arXiv Detail & Related papers (2020-10-24T03:12:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.