Does Explainable Machine Learning Uncover the Black Box in Vision
Applications?
- URL: http://arxiv.org/abs/2112.09898v1
- Date: Sat, 18 Dec 2021 10:37:52 GMT
- Title: Does Explainable Machine Learning Uncover the Black Box in Vision
Applications?
- Authors: Manish Narwaria
- Abstract summary: We argue that the current philosophy behind explainable ML suffers from certain limitations.
We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles.
- Score: 1.0660480034605242
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning (ML) in general and deep learning (DL) in particular has
become an extremely popular tool in several vision applications (like object
detection, super resolution, segmentation, object tracking etc.). Almost in
parallel, the issue of explainability in ML (i.e. the ability to
explain/elaborate the way a trained ML model arrived at its decision) in vision
has also received fairly significant attention from various quarters. However,
we argue that the current philosophy behind explainable ML suffers from certain
limitations, and the resulting explanations may not meaningfully uncover black
box ML models. To elaborate our assertion, we first raise a few fundamental
questions which have not been adequately discussed in the corresponding
literature. We also provide perspectives on how explainablity in ML can benefit
by relying on more rigorous principles in the related areas.
Related papers
- A Survey on Benchmarks of Multimodal Large Language Models [65.87641718350639]
This paper presents a comprehensive review of 200 benchmarks and evaluations for Multimodal Large Language Models (MLLMs)
We focus on (1)perception and understanding, (2)cognition and reasoning, (3)specific domains, (4)key capabilities, and (5)other modalities.
Our key argument is that evaluation should be regarded as a crucial discipline to support the development of MLLMs better.
arXiv Detail & Related papers (2024-08-16T09:52:02Z) - Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge [76.45868419402265]
multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets.
However, the inherent difficulty in explicitly conveying fine-grained or spatially dense information in text, such as masks, poses a challenge for MLLMs.
This paper proposes a new visual prompt approach to integrate fine-grained external knowledge, gleaned from specialized vision models, into MLLMs.
arXiv Detail & Related papers (2024-07-05T17:43:30Z) - Understanding Information Storage and Transfer in Multi-modal Large Language Models [51.20840103605018]
We study how Multi-modal Large Language Models process information in a factual visual question answering task.
Key findings show that these MLLMs rely on self-attention blocks in much earlier layers for information storage.
We introduce MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs.
arXiv Detail & Related papers (2024-06-06T16:35:36Z) - Exploring Perceptual Limitation of Multimodal Large Language Models [57.567868157293994]
We quantitatively study the perception of small visual objects in several state-of-the-art MLLMs.
We identify four independent factors that can contribute to this limitation.
Lower object quality and smaller object size can both independently reduce MLLMs' ability to answer visual questions.
arXiv Detail & Related papers (2024-02-12T03:04:42Z) - Rethinking Interpretability in the Era of Large Language Models [76.1947554386879]
Large language models (LLMs) have demonstrated remarkable capabilities across a wide array of tasks.
The capability to explain in natural language allows LLMs to expand the scale and complexity of patterns that can be given to a human.
These new capabilities raise new challenges, such as hallucinated explanations and immense computational costs.
arXiv Detail & Related papers (2024-01-30T17:38:54Z) - MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception [21.60103376506254]
Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in visual perception and understanding.
These models also suffer from hallucinations, which limit their reliability as AI systems.
This paper aims to define and evaluate the self-awareness of MLLMs in perception.
arXiv Detail & Related papers (2024-01-15T08:19:22Z) - Logic-Based Explainability in Machine Learning [0.0]
The operation of the most successful Machine Learning models is incomprehensible for human decision makers.
In recent years, there have been efforts on devising approaches for explaining ML models.
This paper overviews the ongoing research efforts on computing rigorous model-based explanations of ML models.
arXiv Detail & Related papers (2022-10-24T13:43:07Z) - The games we play: critical complexity improves machine learning [0.0]
We argue that best practice in Machine Learning should be more consistent with critical complexity perspectives than with rationalist, grand narratives.
We identify thirteen 'games' played in the ML community that lend false legitimacy to models, contribute to over-promise and hype about the capabilities of artificial intelligence, lead to models that exacerbate inequality and cause discrimination.
arXiv Detail & Related papers (2022-05-18T13:37:22Z) - Pitfalls of Explainable ML: An Industry Perspective [29.49574255183219]
Explanations sit at the core of desirable attributes of a machine learning (ML) system.
The goal of explainable ML is to intuitively explain the predictions of a ML system, while adhering to the needs to various stakeholders.
arXiv Detail & Related papers (2021-06-14T21:05:05Z) - Survey of explainable machine learning with visual and granular methods
beyond quasi-explanations [0.0]
We focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals.
The paper includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma.
arXiv Detail & Related papers (2020-09-21T23:39:06Z) - An Information-Theoretic Approach to Personalized Explainable Machine
Learning [92.53970625312665]
We propose a simple probabilistic model for the predictions and user knowledge.
We quantify the effect of an explanation by the conditional mutual information between the explanation and prediction.
arXiv Detail & Related papers (2020-03-01T13:06:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.