Does Explainable Machine Learning Uncover the Black Box in Vision Applications?

• URL: http://arxiv.org/abs/2112.09898v1
• Date: Sat, 18 Dec 2021 10:37:52 GMT
• Title: Does Explainable Machine Learning Uncover the Black Box in Vision Applications?
• Authors: Manish Narwaria
• Abstract summary: We argue that the current philosophy behind explainable ML suffers from certain limitations. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles.
• Score: 1.0660480034605242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully uncover black box ML models. To elaborate our assertion, we first raise a few fundamental questions which have not been adequately discussed in the corresponding literature. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles in the related areas.

Related papers

• Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge [76.45868419402265]
  multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets. However, the inherent difficulty in explicitly conveying fine-grained or spatially dense information in text, such as masks, poses a challenge for MLLMs. This paper proposes a new visual prompt approach to integrate fine-grained external knowledge, gleaned from specialized vision models, into MLLMs.
  arXiv  Detail & Related papers  (2024-07-05T17:43:30Z)
• Understanding Information Storage and Transfer in Multi-modal Large Language Models [51.20840103605018]
  We study how Multi-modal Large Language Models process information in a factual visual question answering task. Key findings show that these MLLMs rely on self-attention blocks in much earlier layers for information storage. We introduce MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs.
  arXiv  Detail & Related papers  (2024-06-06T16:35:36Z)
• Exploring Perceptual Limitation of Multimodal Large Language Models [57.567868157293994]
  We quantitatively study the perception of small visual objects in several state-of-the-art MLLMs. We identify four independent factors that can contribute to this limitation. Lower object quality and smaller object size can both independently reduce MLLMs' ability to answer visual questions.
  arXiv  Detail & Related papers  (2024-02-12T03:04:42Z)
• Rethinking Interpretability in the Era of Large Language Models [76.1947554386879]
  Large language models (LLMs) have demonstrated remarkable capabilities across a wide array of tasks. The capability to explain in natural language allows LLMs to expand the scale and complexity of patterns that can be given to a human. These new capabilities raise new challenges, such as hallucinated explanations and immense computational costs.
  arXiv  Detail & Related papers  (2024-01-30T17:38:54Z)
• MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception [21.60103376506254]
  Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in visual perception and understanding. These models also suffer from hallucinations, which limit their reliability as AI systems. This paper aims to define and evaluate the self-awareness of MLLMs in perception.
  arXiv  Detail & Related papers  (2024-01-15T08:19:22Z)
• Democratizing Reasoning Ability: Tailored Learning from Large Language Model [97.4921006089966]
  We propose a tailored learning approach to distill such reasoning ability to smaller LMs. We exploit the potential of LLM as a reasoning teacher by building an interactive multi-round learning paradigm. To exploit the reasoning potential of the smaller LM, we propose self-reflection learning to motivate the student to learn from self-made mistakes.
  arXiv  Detail & Related papers  (2023-10-20T07:50:10Z)
• Logic-Based Explainability in Machine Learning [0.0]
  The operation of the most successful Machine Learning models is incomprehensible for human decision makers. In recent years, there have been efforts on devising approaches for explaining ML models. This paper overviews the ongoing research efforts on computing rigorous model-based explanations of ML models.
  arXiv  Detail & Related papers  (2022-10-24T13:43:07Z)
• The games we play: critical complexity improves machine learning [0.0]
  We argue that best practice in Machine Learning should be more consistent with critical complexity perspectives than with rationalist, grand narratives. We identify thirteen 'games' played in the ML community that lend false legitimacy to models, contribute to over-promise and hype about the capabilities of artificial intelligence, lead to models that exacerbate inequality and cause discrimination.
  arXiv  Detail & Related papers  (2022-05-18T13:37:22Z)
• Pitfalls of Explainable ML: An Industry Perspective [29.49574255183219]
  Explanations sit at the core of desirable attributes of a machine learning (ML) system. The goal of explainable ML is to intuitively explain the predictions of a ML system, while adhering to the needs to various stakeholders.
  arXiv  Detail & Related papers  (2021-06-14T21:05:05Z)
• Survey of explainable machine learning with visual and granular methods beyond quasi-explanations [0.0]
  We focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals. The paper includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma.
  arXiv  Detail & Related papers  (2020-09-21T23:39:06Z)
• An Information-Theoretic Approach to Personalized Explainable Machine Learning [92.53970625312665]
  We propose a simple probabilistic model for the predictions and user knowledge. We quantify the effect of an explanation by the conditional mutual information between the explanation and prediction.
  arXiv  Detail & Related papers  (2020-03-01T13:06:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.