Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling

• URL: http://arxiv.org/abs/2502.10145v1
• Date: Fri, 14 Feb 2025 13:15:21 GMT
• Title: Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling
• Authors: Xinyu Li, Marwa Mahmoud,
• Abstract summary: EU General Data Protection Regulation requires any high-risk AI systems to be sufficiently interpretable. Existing explainable methods often compromise between interpretability and performance. We propose a novel and generalizable framework, namely the Attention-Guided Concept Model (AGCM) AGCM provides learnable conceptual explanations by identifying what concepts that lead to the predictions and where they are observed.
• Score: 5.954573238057435
• License:
• Abstract: In the contemporary era of intelligent connectivity, Affective Computing (AC), which enables systems to recognize, interpret, and respond to human behavior states, has become an integrated part of many AI systems. As one of the most critical components of responsible AI and trustworthiness in all human-centered systems, explainability has been a major concern in AC. Particularly, the recently released EU General Data Protection Regulation requires any high-risk AI systems to be sufficiently interpretable, including biometric-based systems and emotion recognition systems widely used in the affective computing field. Existing explainable methods often compromise between interpretability and performance. Most of them focus only on highlighting key network parameters without offering meaningful, domain-specific explanations to the stakeholders. Additionally, they also face challenges in effectively co-learning and explaining insights from multimodal data sources. To address these limitations, we propose a novel and generalizable framework, namely the Attention-Guided Concept Model (AGCM), which provides learnable conceptual explanations by identifying what concepts that lead to the predictions and where they are observed. AGCM is extendable to any spatial and temporal signals through multimodal concept alignment and co-learning, empowering stakeholders with deeper insights into the model's decision-making process. We validate the efficiency of AGCM on well-established Facial Expression Recognition benchmark datasets while also demonstrating its generalizability on more complex real-world human behavior understanding applications.

Related papers

• Mechanistic understanding and validation of large AI models with SemanticLens [13.712668314238082]
  Unlike human-engineered systems such as aeroplanes, the inner workings of AI models remain largely opaque. This paper introduces SemanticLens, a universal explanation method for neural networks that maps hidden knowledge encoded by components.
  arXiv  Detail & Related papers  (2025-01-09T17:47:34Z)
• Interpretable Rule-Based System for Radar-Based Gesture Sensing: Enhancing Transparency and Personalization in AI [2.99664686845172]
  We introduce MIRA, a transparent and interpretable multi-class rule-based algorithm tailored for radar-based gesture detection. We showcase the system's adaptability through personalized rule sets that calibrate to individual user behavior, offering a user-centric AI experience. Our research underscores MIRA's ability to deliver both high interpretability and performance and emphasizes the potential for broader adoption of interpretable AI in safety-critical applications.
  arXiv  Detail & Related papers  (2024-09-30T16:40:27Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• Emergent Explainability: Adding a causal chain to neural network inference [0.0]
  This position paper presents a theoretical framework for enhancing explainable artificial intelligence (xAI) through emergent communication (EmCom) We explore the novel integration of EmCom into AI systems, offering a paradigm shift from conventional associative relationships between inputs and outputs to a more nuanced, causal interpretation. The paper discusses the theoretical underpinnings of this approach, its potential broad applications, and its alignment with the growing need for responsible and transparent AI systems.
  arXiv  Detail & Related papers  (2024-01-29T02:28:39Z)
• Neurosymbolic Value-Inspired AI (Why, What, and How) [8.946847190099206]
  We propose a neurosymbolic computational framework called Value-Inspired AI (VAI) VAI aims to represent and integrate various dimensions of human values. We offer insights into the current progress made in this direction and outline potential future directions for the field.
  arXiv  Detail & Related papers  (2023-12-15T16:33:57Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Causal Reasoning Meets Visual Representation Learning: A Prospective Study [117.08431221482638]
  Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
  arXiv  Detail & Related papers  (2022-04-26T02:22:28Z)
• An Interactive Explanatory AI System for Industrial Quality Control [0.8889304968879161]
  We aim to extend the defect detection task towards an interactive human-in-the-loop approach. We propose an approach for an interactive support system for classifications in an industrial quality control setting.
  arXiv  Detail & Related papers  (2022-03-17T09:04:46Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)
• Neuro-symbolic Architectures for Context Understanding [59.899606495602406]
  We propose the use of hybrid AI methodology as a framework for combining the strengths of data-driven and knowledge-driven approaches. Specifically, we inherit the concept of neuro-symbolism as a way of using knowledge-bases to guide the learning progress of deep neural networks.
  arXiv  Detail & Related papers  (2020-03-09T15:04:07Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.