Extracting Qualitative Causal Structure with Transformer-Based NLP

• URL: http://arxiv.org/abs/2108.13304v1
• Date: Fri, 20 Aug 2021 20:15:13 GMT
• Title: Extracting Qualitative Causal Structure with Transformer-Based NLP
• Authors: Scott E. Friedman and Ian H. Magnusson and Sonja M. Schmer-Galunder
• Abstract summary: This paper presents a transformer-based NLP architecture that jointly identifies and extracts variables or factors described in language. We demonstrate this approach and include promising results from in two use cases, processing textual inputs from academic publications, news articles, and social media.
• Score: 0.48342038441006785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Qualitative causal relationships compactly express the direction, dependency, temporal constraints, and monotonicity constraints of discrete or continuous interactions in the world. In everyday or academic language, we may express interactions between quantities (e.g., sleep decreases stress), between discrete events or entities (e.g., a protein inhibits another protein's transcription), or between intentional or functional factors (e.g., hospital patients pray to relieve their pain). This paper presents a transformer-based NLP architecture that jointly identifies and extracts (1) variables or factors described in language, (2) qualitative causal relationships over these variables, and (3) qualifiers and magnitudes that constrain these causal relationships. We demonstrate this approach and include promising results from in two use cases, processing textual inputs from academic publications, news articles, and social media.

Related papers

• Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
  We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2023-08-24T16:06:36Z)
• CIParsing: Unifying Causality Properties into Multiple Human Parsing [82.32620538918812]
  Existing methods of multiple human parsing (MHP) apply statistical models to acquire underlying associations between images and labeled body parts. We present a causality inspired parsing paradigm termed CIParsing, which follows fundamental causal principles involving two causal properties for human parsing. The CIParsing is designed in a plug-and-play fashion and can be integrated into any existing MHP models.
  arXiv  Detail & Related papers  (2023-08-23T15:56:26Z)
• Nonlinearity, Feedback and Uniform Consistency in Causal Structural Learning [0.8158530638728501]
  Causal Discovery aims to find automated search methods for learning causal structures from observational data. This thesis focuses on two questions in causal discovery: (i) providing an alternative definition of k-Triangle Faithfulness that (i) is weaker than strong faithfulness when applied to the Gaussian family of distributions, and (ii) under the assumption that the modified version of Strong Faithfulness holds.
  arXiv  Detail & Related papers  (2023-08-15T01:23:42Z)
• A Causal Framework for Decomposing Spurious Variations [68.12191782657437]
  We develop tools for decomposing spurious variations in Markovian and Semi-Markovian models. We prove the first results that allow a non-parametric decomposition of spurious effects. The described approach has several applications, ranging from explainable and fair AI to questions in epidemiology and medicine.
  arXiv  Detail & Related papers  (2023-06-08T09:40:28Z)
• Causal Deep Learning [77.49632479298745]
  Causality has the potential to transform the way we solve real-world problems. But causality often requires crucial assumptions which cannot be tested in practice. We propose a new way of thinking about causality -- we call this causal deep learning.
  arXiv  Detail & Related papers  (2023-03-03T19:19:18Z)
• Uninformative Input Features and Counterfactual Invariance: Two Perspectives on Spurious Correlations in Natural Language [19.416033815407804]
  Spurious correlations are a threat to the trustworthiness of natural language processing systems. This paper analyzes three distinct conditions that can give rise to feature-label correlations in a simple PCFG. Because input features will be individually correlated with labels in all but very rare circumstances, domain knowledge must be applied to identify spurious correlations that pose genuine robustness threats.
  arXiv  Detail & Related papers  (2022-04-09T14:46:39Z)
• From Unstructured Text to Causal Knowledge Graphs: A Transformer-Based Approach [4.133048890906828]
  This paper presents a transformer-based NLP architecture that jointly extracts knowledge graphs including variables or factors described in language. We provide evidence of its accurate knowledge graph extraction in real-world domains and the practicality of its resulting knowledge graphs for cognitive systems that perform graph-based reasoning.
  arXiv  Detail & Related papers  (2022-02-23T20:29:55Z)
• Text-based NP Enrichment [51.403543011094975]
  We establish the task of text-based NP enrichment (TNE), that is, enriching each NP with all the preposition-mediated relations that hold between this and the other NPs in the text. Humans recover such relations seamlessly, while current state-of-the-art models struggle with them due to the implicit nature of the problem. We build the first large-scale dataset for the problem, provide the formal framing and scope of annotation, analyze the data, and report the result of fine-tuned neural language models on the task.
  arXiv  Detail & Related papers  (2021-09-24T17:23:25Z)
• Linguistic dependencies and statistical dependence [76.89273585568084]
  We use pretrained language models to estimate probabilities of words in context. We find that maximum-CPMI trees correspond to linguistic dependencies more often than trees extracted from non-contextual PMI estimate.
  arXiv  Detail & Related papers  (2021-04-18T02:43:37Z)
• Learning Informative Representations of Biomedical Relations with Latent Variable Models [2.4366811507669115]
  We propose a latent variable model with an arbitrarily flexible distribution to represent the relation between an entity pair. We demonstrate that our model achieves results competitive with strong baselines for both tasks while having fewer parameters and being significantly faster to train.
  arXiv  Detail & Related papers  (2020-11-20T08:56:31Z)
• Estimating Causal Effects with the Neural Autoregressive Density Estimator [6.59529078336196]
  We use neural autoregressive density estimators to estimate causal effects within the Pearl's do-calculus framework. We show that the approach can retrieve causal effects from non-linear systems without explicitly modeling the interactions between the variables.
  arXiv  Detail & Related papers  (2020-08-17T13:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.