Unsupervised Explanation Generation for Machine Reading Comprehension

• URL: http://arxiv.org/abs/2011.06737v1
• Date: Fri, 13 Nov 2020 02:58:55 GMT
• Title: Unsupervised Explanation Generation for Machine Reading Comprehension
• Authors: Yiming Cui, Ting Liu, Shijin Wang, Guoping Hu
• Abstract summary: We propose a self-explainable framework for the machine reading comprehension task. The proposed system tries to use less passage information and achieve similar results compared to the system that uses the whole passage. To evaluate the explainability, we compared our approach with the traditional attention mechanism in human evaluations and found that the proposed system has a notable advantage over the latter one.
• Score: 36.182335120466895
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: With the blooming of various Pre-trained Language Models (PLMs), Machine Reading Comprehension (MRC) has embraced significant improvements on various benchmarks and even surpass human performances. However, the existing works only target on the accuracy of the final predictions and neglect the importance of the explanations for the prediction, which is a big obstacle when utilizing these models in real-life applications to convince humans. In this paper, we propose a self-explainable framework for the machine reading comprehension task. The main idea is that the proposed system tries to use less passage information and achieve similar results compared to the system that uses the whole passage, while the filtered passage will be used as explanations. We carried out experiments on three multiple-choice MRC datasets, and found that the proposed system could achieve consistent improvements over baseline systems. To evaluate the explainability, we compared our approach with the traditional attention mechanism in human evaluations and found that the proposed system has a notable advantage over the latter one.

Related papers

• Controlled Training Data Generation with Diffusion Models [48.123126522294015]
  We present a method to control a text-to-image generative model to produce training data specifically "useful" for supervised learning. We develop an automated closed-loop system which involves two feedback mechanisms.
  arXiv  Detail & Related papers  (2024-03-22T15:59:24Z)
• Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms [15.542532119818794]
  Two prevalent paradigms, spot-checking and peer prediction, enable the design of mechanisms to evaluate and incentivize high-quality data from human labelers. We show that two of these metrics are actually the same within certain contexts and explain the divergence of the third. We present two approaches to compute spot check equivalence in various contexts, where simulation results verify the effectiveness of our proposed metric.
  arXiv  Detail & Related papers  (2024-02-21T06:57:07Z)
• Machine-Learning Solutions for the Analysis of Single-Particle Diffusion Trajectories [0.0]
  We provide an overview over recently introduced methods in machine-learning for diffusive time series. We focus on means to include uncertainty estimates and feature-based approaches, both improving interpretability and providing concrete insight into the learning process of the machine.
  arXiv  Detail & Related papers  (2023-08-18T09:29:29Z)
• Counterfactual Explanation via Search in Gaussian Mixture Distributed Latent Space [19.312306559210125]
  Counterfactual Explanations (CEs) are an important tool in Algorithmic Recourse for addressing two questions. guiding the user's interaction with AI systems by proposing easy-to-understand explanations is essential for the trustworthy adoption and long-term acceptance of AI systems. We introduce a new method to generate CEs for a pre-trained binary classifier by first shaping the latent space of an autoencoder to be a mixture of Gaussian distributions.
  arXiv  Detail & Related papers  (2023-07-25T10:21:26Z)
• Towards Explainable Evaluation Metrics for Machine Translation [32.69015745456696]
  We identify key properties as well as key goals of explainable machine translation metrics. We discuss the latest state-of-the-art approaches to explainable metrics based on generative models such as ChatGPT and GPT4.
  arXiv  Detail & Related papers  (2023-06-22T17:07:57Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Explain and Predict, and then Predict Again [6.865156063241553]
  We propose ExPred, that uses multi-task learning in the explanation generation phase effectively trading-off explanation and prediction losses. We conduct an extensive evaluation of our approach on three diverse language datasets.
  arXiv  Detail & Related papers  (2021-01-11T19:36:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.