Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions

• URL: http://arxiv.org/abs/2604.11730v2
• Date: Tue, 14 Apr 2026 11:00:18 GMT
• Title: Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions
• Abstract summary: Ambivalence and hesitancy (A/H) play a primary role for individuals to delay, avoid, or abandon health interventions.<n>While experts can be trained to recognize A/H, integrating them into digital health interventions is costly and less effective.<n>Here, we explore the application of deep learning models for A/H recognition in videos, a multi-modal task by nature.
• Score: 51.74800196847922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Using behavioural science, health interventions focus on behaviour change by providing a framework to help patients acquire and maintain healthy habits that improve medical outcomes. In-person interventions are costly and difficult to scale, especially in resource-limited regions. Digital health interventions offer a cost-effective approach, potentially supporting independent living and self-management. Automating such interventions, especially through machine learning, has gained considerable attention recently. Ambivalence and hesitancy (A/H) play a primary role for individuals to delay, avoid, or abandon health interventions. A/H are subtle and conflicting emotions that place a person in a state between positive and negative evaluations of a behaviour, or between acceptance and refusal to engage in it. They manifest as affective inconsistency across modalities or within a modality, such as language, facial, vocal expressions, and body language. While experts can be trained to recognize A/H, integrating them into digital health interventions is costly and less effective. Automatic A/H recognition is therefore critical for the personalization and cost-effectiveness of digital health interventions. Here, we explore the application of deep learning models for A/H recognition in videos, a multi-modal task by nature. In particular, this paper covers three learning setups: supervised learning, unsupervised domain adaptation for personalization, and zero-shot inference via large language models (LLMs). Our experiments are conducted on the unique and recently published BAH video dataset for A/H recognition. Our results show limited performance, suggesting that more adapted multi-modal models are required for accurate A/H recognition. Better methods for modeling spatio-temporal and multimodal fusion are necessary to leverage conflicts within/across modalities.

Related papers

• Generative Experiences for Digital Mental Health Interventions: Evidence from a Randomized Study [56.17549485306432]
  GUIDE is a system that generates personalized intervention content and multimodal interaction structure.<n>In a preregistered study with N = 237 participants, GUIDE significantly reduced stress and improved the user experience.<n>This work lays the foundation for interventions that dynamically shape how support is experienced and enacted in digital settings.
  arXiv  Detail & Related papers  (2026-04-08T19:59:28Z)
• Airway Skill Assessment with Spatiotemporal Attention Mechanisms Using Human Gaze [2.125763116614213]
  Airway management skills are critical in emergency medicine and are typically assessed through subjective evaluation.<n>This paper proposes a machine learning-based approach for assessing airway skills, specifically endotracheal intubation (ETI)<n>The proposed system leverages an attention mechanism guided by the human gaze to enhance the recognition of successful and unsuccessful ETI procedures.
  arXiv  Detail & Related papers  (2025-06-24T04:40:58Z)
• GEMeX-RMCoT: An Enhanced Med-VQA Dataset for Region-Aware Multimodal Chain-of-Thought Reasoning [60.03671205298294]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability.<n>This work first proposes a Region-Aware Multimodal Chain-of-Thought dataset, in which the process of producing an answer is preceded by a sequence of intermediate reasoning steps.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation [55.2739790399209]
  We present AutoMedEval, an open-sourced automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs.<n>The overarching objective of AutoMedEval is to assess the quality of responses produced by diverse models, aspiring to significantly reduce the dependence on human evaluation.
  arXiv  Detail & Related papers  (2025-05-17T07:44:54Z)
• Predicting Treatment Response in Body Dysmorphic Disorder with Interpretable Machine Learning [0.4604003661048266]
  Body Dysmorphic Disorder (BDD) is a highly prevalent and frequently underdiagnosed condition.<n>We employ multiple machine learning approaches to predict treatment outcomes.<n>Treatment credibility emerged as the most potent predictor.
  arXiv  Detail & Related papers  (2025-03-13T17:39:10Z)
• AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women [32.514036618021244]
  Major Depressive Disorder and anxiety disorders affect millions globally, contributing significantly to the burden of mental health issues.<n>Early screening is crucial for effective intervention, as timely identification of mental health issues can significantly improve treatment outcomes.<n>This study explores the potential of AI models for ubiquitous depression-anxiety screening given face-centric selfies.
  arXiv  Detail & Related papers  (2024-10-07T19:34:25Z)
• Explainable Artificial Intelligence for Quantifying Interfering and High-Risk Behaviors in Autism Spectrum Disorder in a Real-World Classroom Environment Using Privacy-Preserving Video Analysis [2.286750720737624]
  We show the latest advances in video-based group activity recognition techniques can quantify behaviors in ASD in real-world activities. Our explainable model could detect the episode of problem behaviors with a 77% F1-score and capture distinctive behavior features in different types of behaviors in ASD.
  arXiv  Detail & Related papers  (2024-07-31T15:37:52Z)
• Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration [54.76511683427566]
  This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information. A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction. The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures.
  arXiv  Detail & Related papers  (2024-02-16T05:16:20Z)
• I am Only Happy When There is Light: The Impact of Environmental Changes on Affective Facial Expressions Recognition [65.69256728493015]
  We study the impact of different image conditions on the recognition of arousal from human facial expressions. Our results show how the interpretation of human affective states can differ greatly in either the positive or negative direction.
  arXiv  Detail & Related papers  (2022-10-28T16:28:26Z)
• In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains [22.92165116962952]
  In-bed human posture estimation provides important health-related metrics with potential value in medical condition assessments. We propose a multi-modal conditional variational autoencoder (MC-VAE) capable of reconstructing features from missing modalities seen during training. We demonstrate that body positions can be effectively recognized from the available modality, achieving on par results with baseline models.
  arXiv  Detail & Related papers  (2021-11-30T04:56:16Z)
• Non-contact Pain Recognition from Video Sequences with Remote Physiological Measurements Prediction [53.03469655641418]
  We present a novel multi-task learning framework which encodes both appearance changes and physiological cues in a non-contact manner for pain recognition. We establish the state-of-the-art performance of non-contact pain recognition on publicly available pain databases.
  arXiv  Detail & Related papers  (2021-05-18T20:47:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.