Causal BERT : Language models for causality detection between events expressed in text

• URL: http://arxiv.org/abs/2012.05453v2
• Date: Thu, 28 Jan 2021 21:15:26 GMT
• Title: Causal BERT : Language models for causality detection between events expressed in text
• Authors: Vivek Khetan, Roshni Ramnani, Mayuresh Anand, Shubhashis Sengupta and Andrew E.Fano
• Abstract summary: Causality understanding between events is helpful in many areas, including health care, business risk management and finance. "Cause-Effect" relationships between natural language events continues to remain a challenge simply because it is often expressed implicitly. Our proposed methods achieve the state-of-art performance in three different data distributions and can be leveraged for extraction of a causal diagram.
• Score: 1.0756038762528868
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Causality understanding between events is a critical natural language processing task that is helpful in many areas, including health care, business risk management and finance. On close examination, one can find a huge amount of textual content both in the form of formal documents or in content arising from social media like Twitter, dedicated to communicating and exploring various types of causality in the real world. Recognizing these "Cause-Effect" relationships between natural language events continues to remain a challenge simply because it is often expressed implicitly. Implicit causality is hard to detect through most of the techniques employed in literature and can also, at times be perceived as ambiguous or vague. Also, although well-known datasets do exist for this problem, the examples in them are limited in the range and complexity of the causal relationships they depict especially when related to implicit relationships. Most of the contemporary methods are either based on lexico-semantic pattern matching or are feature-driven supervised methods. Therefore, as expected these methods are more geared towards handling explicit causal relationships leading to limited coverage for implicit relationships and are hard to generalize. In this paper, we investigate the language model's capabilities for causal association among events expressed in natural language text using sentence context combined with event information, and by leveraging masked event context with in-domain and out-of-domain data distribution. Our proposed methods achieve the state-of-art performance in three different data distributions and can be leveraged for extraction of a causal diagram and/or building a chain of events from unstructured text.

Related papers

• Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs [61.796960984541464]
  We present COM2 (COMplex COMmonsense), a new dataset created by sampling logical queries. We verbalize them using handcrafted rules and large language models into multiple-choice and text generation questions. Experiments show that language models trained on COM2 exhibit significant improvements in complex reasoning ability.
  arXiv  Detail & Related papers  (2024-03-12T08:13:52Z)
• Recognizing Conditional Causal Relationships about Emotions and Their Corresponding Conditions [37.16991100831717]
  We propose a new task to determine whether an input pair of emotion and cause has a valid causal relationship under different contexts. We use negative sampling to construct the final dataset to balance the number of documents with and without causal relationships.
  arXiv  Detail & Related papers  (2023-11-28T07:47:25Z)
• Causal schema induction for knowledge discovery [21.295680010103602]
  We present Torquestra, a dataset of text-graph-schema units integrating temporal, event, and causal structures. We benchmark our dataset on three knowledge discovery tasks, building and evaluating models for each. Results show that systems that harness causal structure are effective at identifying texts sharing similar causal meaning components.
  arXiv  Detail & Related papers  (2023-03-27T16:55:49Z)
• Contextual information integration for stance detection via cross-attention [59.662413798388485]
  Stance detection deals with identifying an author's stance towards a target. Most existing stance detection models are limited because they do not consider relevant contextual information. We propose an approach to integrate contextual information as text.
  arXiv  Detail & Related papers  (2022-11-03T15:04:29Z)
• Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences [6.994580267603235]
  This work contributes to the relatively untrodden path of what is required in data for models to capture meaningful representations of natural language. This entails evaluating how well English and Spanish semantic spaces capture a particular type of relational knowledge.
  arXiv  Detail & Related papers  (2022-05-16T12:09:40Z)
• Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge [59.22170796793179]
  Transformers Language Models (TLMs) were tested on a benchmark for the textitdynamic estimation of thematic fit Our results show that TLMs can reach performances that are comparable to those achieved by SDM. However, additional analysis consistently suggests that TLMs do not capture important aspects of event knowledge.
  arXiv  Detail & Related papers  (2021-07-22T20:52:26Z)
• Event Argument Extraction using Causal Knowledge Structures [9.56216681584111]
  Event Argument extraction refers to the task of extracting structured information from unstructured text for a particular event of interest. Most of the existing works model this task at a sentence level, restricting the context to a local scope. We propose an external knowledge aided approach to infuse document-level event information to aid the extraction of complex event arguments.
  arXiv  Detail & Related papers  (2021-05-02T13:59:07Z)
• Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
  We argue that for complex language understanding tasks, all simple feature correlations are spurious. We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
  arXiv  Detail & Related papers  (2021-04-17T21:34:10Z)
• Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection [62.071938098215085]
  We focus on the Commongen benchmark, wherein the aim is to generate a plausible sentence for a given set of input concepts. We propose strategies for enhancing the semantic correctness of the generated text.
  arXiv  Detail & Related papers  (2020-12-19T23:23:40Z)
• Learning Causal Bayesian Networks from Text [15.739271050022891]
  Causal relationships form the basis for reasoning and decision-making in Artificial Intelligence systems. We propose a method for automatic inference of causal relationships from human written language at conceptual level.
  arXiv  Detail & Related papers  (2020-11-26T03:57:56Z)
• Joint Constrained Learning for Event-Event Relation Extraction [94.3499255880101]
  We propose a joint constrained learning framework for modeling event-event relations. Specifically, the framework enforces logical constraints within and across multiple temporal and subevent relations. We show that our joint constrained learning approach effectively compensates for the lack of jointly labeled data.
  arXiv  Detail & Related papers  (2020-10-13T22:45:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.