Explaining Image Classification with Visual Debates

• URL: http://arxiv.org/abs/2210.09015v2
• Date: Tue, 23 May 2023 09:58:18 GMT
• Title: Explaining Image Classification with Visual Debates
• Authors: Avinash Kori, Ben Glocker, Francesca Toni
• Abstract summary: We propose a novel debate framework for understanding and explaining a continuous image classifier's reasoning for making a particular prediction. Our framework encourages players to put forward diverse arguments during the debates, picking up the reasoning trails missed by their opponents. We demonstrate and evaluate (a practical realization) our Visual Debates on the geometric SHAPE and MNIST datasets.
• Score: 26.76139301708958
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: An effective way to obtain different perspectives on any given topic is by conducting a debate, where participants argue for and against the topic. Here, we propose a novel debate framework for understanding and explaining a continuous image classifier's reasoning for making a particular prediction by modeling it as a multiplayer sequential zero-sum debate game. The contrastive nature of our framework encourages players to learn to put forward diverse arguments during the debates, picking up the reasoning trails missed by their opponents and highlighting any uncertainties in the classifier. Specifically, in our proposed setup, players propose arguments, drawn from the classifier's discretized latent knowledge, to support or oppose the classifier's decision. The resulting Visual Debates collect supporting and opposing features from the discretized latent space of the classifier, serving as explanations for the internal reasoning of the classifier towards its predictions. We demonstrate and evaluate (a practical realization of) our Visual Debates on the geometric SHAPE and MNIST datasets and on the high-resolution animal faces (AFHQ) dataset, along standard evaluation metrics for explanations (i.e. faithfulness and completeness) and novel, bespoke metrics for visual debates as explanations (consensus and split ratio).

Related papers

• A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning [53.35861580821777]
  The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG) In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts.
  arXiv  Detail & Related papers  (2024-03-22T06:03:07Z)
• Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM [51.43102092480804]
  Debatrix is an automated debate judge based on Large Language Models (LLMs) To align with real-world debate scenarios, we introduced the PanelBench benchmark, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation.
  arXiv  Detail & Related papers  (2024-03-12T18:19:47Z)
• Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation [62.069374456021016]
  We present the ArgTersely benchmark for sentence-level counter-argument generation. We also propose Arg-LlaMA for generating high-quality counter-argument.
  arXiv  Detail & Related papers  (2023-12-21T06:51:34Z)
• Indicative Summarization of Long Discussions [37.80285705350554]
  This paper presents a novel unsupervised approach using large language models (LLMs) to generate indicative summaries for long discussions. Our approach first clusters argument sentences, generates cluster labels as abstractive summaries, and classifies the generated cluster labels into argumentation frames. Based on an extensively optimized prompt engineering approach, we evaluate 19LLMs for generative cluster labeling and frame classification.
  arXiv  Detail & Related papers  (2023-11-03T12:44:59Z)
• A Unifying Framework for Learning Argumentation Semantics [50.69905074548764]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way. Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion [52.49981085431061]
  Enhancing people's ability to write persuasive arguments could contribute to the effectiveness and civility in online communication. We derived four design goals for a tool that helps users improve the persuasiveness of arguments in online discussions. Persua is an interactive visual system that provides example-based guidance on persuasive strategies to enhance the persuasiveness of arguments.
  arXiv  Detail & Related papers  (2022-04-16T08:07:53Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)
• The Unfolding Structure of Arguments in Online Debates: The case of a No-Deal Brexit [0.0]
  We propose a five-step methodology to extract, categorize and explore the latent argumentation structures of online debates. Using Twitter data about a "no-deal" Brexit, we focus on the expected effects in case of materialisation of this event. Results show that the proposed methodology can be employed to perform a statistical rhetorics analysis of debates.
  arXiv  Detail & Related papers  (2021-03-09T12:29:43Z)
• Exploring the Role of Argument Structure in Online Debate Persuasion [39.74040217761505]
  We investigate the role of discourse structure of the arguments from online debates in their persuasiveness. We find that argument structure features play an essential role in achieving the better predictive performance.
  arXiv  Detail & Related papers  (2020-10-07T17:34:50Z)
• AMPERSAND: Argument Mining for PERSuAsive oNline Discussions [41.06165177604387]
  We propose a computational model for argument mining in online persuasive discussion forums. Our approach relies on identifying relations between components of arguments in a discussion thread. Our models obtain significant improvements compared to recent state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-04-30T10:33:40Z)
• Reasoning on Knowledge Graphs with Debate Dynamics [27.225048123690243]
  We propose a novel method for automatic reasoning on knowledge graphs based on debate dynamics. The main idea is to frame the task of triple classification as a debate game between two reinforcement learning agents. We benchmark our method on the triple classification and link prediction task.
  arXiv  Detail & Related papers  (2020-01-02T14:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.