A Multimodal XAI Framework for Trustworthy CNNs and Bias Detection in Deep Representation Learning

• URL: http://arxiv.org/abs/2510.12957v1
• Date: Tue, 14 Oct 2025 20:06:09 GMT
• Title: A Multimodal XAI Framework for Trustworthy CNNs and Bias Detection in Deep Representation Learning
• Authors: Noor Islam S. Mohammad,
• Abstract summary: We propose a novel framework that unifies attention-augmented feature fusion, Grad-CAM++-based local explanations, and a Reveal-to-Revise feedback loop for bias detection and mitigation.<n>Our approach achieves 93.2% classification accuracy, 91.6% F1-score, and 78.1% explanation fidelity (IoU-XAI)<n>Our work bridges the gap between performance, transparency, and fairness, highlighting a practical pathway for trustworthy AI in sensitive domains.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard benchmark datasets, such as MNIST, often fail to expose latent biases and multimodal feature complexities, limiting the trustworthiness of deep neural networks in high-stakes applications. We propose a novel multimodal Explainable AI (XAI) framework that unifies attention-augmented feature fusion, Grad-CAM++-based local explanations, and a Reveal-to-Revise feedback loop for bias detection and mitigation. Evaluated on multimodal extensions of MNIST, our approach achieves 93.2% classification accuracy, 91.6% F1-score, and 78.1% explanation fidelity (IoU-XAI), outperforming unimodal and non-explainable baselines. Ablation studies demonstrate that integrating interpretability with bias-aware learning enhances robustness and human alignment. Our work bridges the gap between performance, transparency, and fairness, highlighting a practical pathway for trustworthy AI in sensitive domains.

Related papers

• Multi-layer Cross-Attention is Provably Optimal for Multi-modal In-context Learning [7.67220299822976]
  We introduce a mathematically tractable framework for studying multi-modal learning and explore when transformer-like architectures can recover Bayes-optimal performance in-context.<n>Our results underscore the benefits of depth for in-context learning and establish the provable utility of cross-attention for multi-modal distributions.
  arXiv  Detail & Related papers  (2026-02-04T18:57:30Z)
• From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation [59.27094165576015]
  We propose a novel learning paradigm (UniMod) that transitions from sparse decision-making to dense reasoning traces.<n>By constructing structured trajectories encompassing evidence grounding, modality assessment, risk mapping, policy decision, and response generation, we reformulate monolithic decision tasks into a multi-dimensional boundary learning process.<n>We introduce specialized optimization strategies to decouple task-specific parameters and rebalance training dynamics, effectively resolving interference between diverse objectives in multi-task learning.
  arXiv  Detail & Related papers  (2026-01-28T09:29:40Z)
• Enhancing the QA Model through a Multi-domain Debiasing Framework [1.7802147489386633]
  This study evaluates the ELECTRA-small model on the Stanford Question Answering dataset (SQuAD) v1.1 and adversarial datasets AddSent and AddOneSent.<n>We develop a multi-domain debiasing framework incorporating knowledge distillation, debiasing techniques, and domain expansion.
  arXiv  Detail & Related papers  (2026-01-01T08:39:07Z)
• Multimodal Detection of Fake Reviews using BERT and ResNet-50 [0.0]
  A robust multimodal fake review detection framework is proposed, integrating textual features encoded with BERT and visual features extracted using ResNet-50.<n> Experimental results indicate that the multimodal model outperforms unimodal baselines, achieving an F1-score of 0.934 on the test set.<n>This study demonstrates the critical role of multimodal learning in safeguarding digital trust and offers a scalable solution for content moderation.
  arXiv  Detail & Related papers  (2025-10-24T01:24:53Z)
• Multiscale Adaptive Conflict-Balancing Model For Multimedia Deepfake Detection [4.849608823153888]
  multimodal detection methods remain limited by unbalanced learning between modalities.<n>We propose an Audio-Visual Joint Learning Method (MACB-DF) to better mitigate modality conflicts and neglect.<n>Our method exhibits superior cross-dataset generalization capabilities, with absolute improvements of 8.0% and 7.7% in ACC scores over the previous best-performing approach.
  arXiv  Detail & Related papers  (2025-05-19T11:01:49Z)
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models [71.36392373876505]
  We introduce MMIE, a large-scale benchmark for evaluating interleaved multimodal comprehension and generation in Large Vision-Language Models (LVLMs)<n>MMIE comprises 20K meticulously curated multimodal queries, spanning 3 categories, 12 fields, and 102 subfields, including mathematics, coding, physics, literature, health, and arts.<n>It supports both interleaved inputs and outputs, offering a mix of multiple-choice and open-ended question formats to evaluate diverse competencies.
  arXiv  Detail & Related papers  (2024-10-14T04:15:00Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• When Can You Trust Your Explanations? A Robustness Analysis on Feature Importances [42.36530107262305]
  robustness of explanations plays a central role in ensuring trust in both the system and the provided explanation.<n>We propose a novel approach to analyse the robustness of neural network explanations to non-adversarial perturbations.<n>We additionally present an ensemble method to aggregate various explanations, showing how merging explanations can be beneficial for both understanding the model's decision and evaluating the robustness.
  arXiv  Detail & Related papers  (2024-06-20T14:17:57Z)
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks [3.9584068556746246]
  We introduce a novel framework designed to enhance the explainability of deep networks through a dual focus on maximizing both accuracy and comprehensibility in explanations.<n>Our framework integrates outputs from multiple established XAI methods and leverages a non-linear neural network model, termed the 'explanation,' to construct a unified, optimal explanation.
  arXiv  Detail & Related papers  (2024-05-16T11:49:08Z)
• Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching [53.05954114863596]
  We propose a brand-new Deep Boosting Learning (DBL) algorithm for image-text matching. An anchor branch is first trained to provide insights into the data properties. A target branch is concurrently tasked with more adaptive margin constraints to further enlarge the relative distance between matched and unmatched samples.
  arXiv  Detail & Related papers  (2024-04-28T08:44:28Z)
• The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus [10.135749005469686]
  One of the unsolved challenges in the field of Explainable AI (XAI) is determining how to most reliably estimate the quality of an explanation method. We address this issue through a meta-evaluation of different quality estimators in XAI. Our novel framework, MetaQuantus, analyses two complementary performance characteristics of a quality estimator.
  arXiv  Detail & Related papers  (2023-02-14T18:59:02Z)
• Generalizability of Adversarial Robustness Under Distribution Shifts [57.767152566761304]
  We take a first step towards investigating the interplay between empirical and certified adversarial robustness on one hand and domain generalization on another. We train robust models on multiple domains and evaluate their accuracy and robustness on an unseen domain. We extend our study to cover a real-world medical application, in which adversarial augmentation significantly boosts the generalization of robustness with minimal effect on clean data accuracy.
  arXiv  Detail & Related papers  (2022-09-29T18:25:48Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.