Evaluating Explanation Methods for Vision-and-Language Navigation
- URL: http://arxiv.org/abs/2310.06654v1
- Date: Tue, 10 Oct 2023 14:22:56 GMT
- Title: Evaluating Explanation Methods for Vision-and-Language Navigation
- Authors: Guanqi Chen, Lei Yang, Guanhua Chen, Jia Pan
- Abstract summary: We propose a new erasure-based evaluation pipeline to measure the step-wise textual explanation in the sequential decision-making setting.
We evaluate several explanation methods for two representative VLN models on two popular VLN datasets.
- Score: 26.607847422870464
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The ability to navigate robots with natural language instructions in an
unknown environment is a crucial step for achieving embodied artificial
intelligence (AI). With the improving performance of deep neural models
proposed in the field of vision-and-language navigation (VLN), it is equally
interesting to know what information the models utilize for their
decision-making in the navigation tasks. To understand the inner workings of
deep neural models, various explanation methods have been developed for
promoting explainable AI (XAI). But they are mostly applied to deep neural
models for image or text classification tasks and little work has been done in
explaining deep neural models for VLN tasks. In this paper, we address these
problems by building quantitative benchmarks to evaluate explanation methods
for VLN models in terms of faithfulness. We propose a new erasure-based
evaluation pipeline to measure the step-wise textual explanation in the
sequential decision-making setting. We evaluate several explanation methods for
two representative VLN models on two popular VLN datasets and reveal valuable
findings through our experiments.
Related papers
- Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process.
We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
arXiv Detail & Related papers (2024-01-11T18:57:17Z) - Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - Explaining Explainability: Towards Deeper Actionable Insights into Deep
Learning through Second-order Explainability [70.60433013657693]
Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level.
We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
arXiv Detail & Related papers (2023-06-14T23:24:01Z) - Benchmarking Faithfulness: Towards Accurate Natural Language
Explanations in Vision-Language Tasks [0.0]
Natural language explanations (NLEs) promise to enable the communication of a model's decision-making in an easily intelligible way.
While current models successfully generate convincing explanations, it is an open question how well the NLEs actually represent the reasoning process of the models.
We propose three faithfulness metrics: Attribution-Similarity, NLE-Sufficiency, and NLE-Comprehensiveness.
arXiv Detail & Related papers (2023-04-03T08:24:10Z) - On the Explainability of Natural Language Processing Deep Models [3.0052400859458586]
Methods have been developed to address the challenges and present satisfactory explanations on Natural Language Processing (NLP) models.
Motivated to democratize ExAI methods in the NLP field, we present in this work a survey that studies model-agnostic as well as model-specific explainability methods on NLP models.
arXiv Detail & Related papers (2022-10-13T11:59:39Z) - Deep Learning Reproducibility and Explainable AI (XAI) [9.13755431537592]
The nondeterminism of Deep Learning (DL) training algorithms and its influence on the explainability of neural network (NN) models are investigated.
To discuss the issue, two convolutional neural networks (CNN) have been trained and their results compared.
arXiv Detail & Related papers (2022-02-23T12:06:20Z) - Multi-Semantic Image Recognition Model and Evaluating Index for
explaining the deep learning models [31.387124252490377]
We first propose a multi-semantic image recognition model, which enables human beings to understand the decision-making process of the neural network.
We then presents a new evaluation index, which can quantitatively assess the model interpretability.
This paper also exhibits the relevant baseline performance with current state-of-the-art deep learning models.
arXiv Detail & Related papers (2021-09-28T07:18:05Z) - e-ViL: A Dataset and Benchmark for Natural Language Explanations in
Vision-Language Tasks [52.918087305406296]
We introduce e-ViL, a benchmark for evaluate explainable vision-language tasks.
We also introduce e-SNLI-VE, the largest existing dataset with NLEs.
We propose a new model that combines UNITER, which learns joint embeddings of images and text, and GPT-2, a pre-trained language model.
arXiv Detail & Related papers (2021-05-08T18:46:33Z) - Explainability in Deep Reinforcement Learning [68.8204255655161]
We review recent works in the direction to attain Explainable Reinforcement Learning (XRL)
In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
arXiv Detail & Related papers (2020-08-15T10:11:42Z) - Soft Expert Reward Learning for Vision-and-Language Navigation [94.86954695912125]
Vision-and-Language Navigation (VLN) requires an agent to find a specified spot in an unseen environment by following natural language instructions.
We introduce a Soft Expert Reward Learning (SERL) model to overcome the reward engineering designing and generalisation problems of the VLN task.
arXiv Detail & Related papers (2020-07-21T14:17:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.