Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

• URL: http://arxiv.org/abs/2606.28104v1
• Date: Fri, 26 Jun 2026 14:00:05 GMT
• Title: Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training
• Abstract summary: Vision-based assessment can provide convenient and cost-effective evaluation in Traditional Chinese Medicine (TCM) rehabilitation training.<n>We propose CME-AQA, a cross-view, multimodal vision-based assessment framework that integrates visual-pose fusion to enhance understanding of environmental context.
• Score: 14.830434505973438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-based assessment can provide convenient and cost-effective evaluation in Traditional Chinese Medicine (TCM) rehabilitation training, where action quality assessment (AQA) from computer vision offers a promising solution. Existing automatic AQA frameworks for physical therapy typically rely on skeletal data captured from a single viewpoint, which is inefficient for TCM techniques such as acupuncture or Tuina that involve dense hand self-occlusion and complex hand-object interactions. To address these challenges, we propose CME-AQA, a cross-view, multimodal vision-based assessment framework that integrates visual-pose fusion to enhance understanding of environmental context and leverages both first-person and third-person videos during training to improve inference robustness. We collected two dual-view datasets, TCM-AQA61-A (Acupuncture) and TCM-AQA61-T (Tuina), each containing synchronized first-person and third-person recordings of 61 subjects with expert annotations. Experimental results show that our approach achieves superior or comparable mean performance against competitive baselines, achieving over 10% relative improvement in weighted F1 over the best competing method on key rating tasks such as Needle Depth and Quick Needle Insertion, while also reducing mean absolute error in quantitative measures such as insertion time and manipulation frequency. Testing on a CPR dataset further demonstrates comparable performance on several posture-based criteria, suggesting applicability to related structured simulated clinical skill assessments where participant motion is central to evaluation. Overall, CME-AQA enhances assessment accuracy for structured TCM rehabilitation training and facilitates more convenient and effective training-oriented skill evaluation.

Related papers

• A Multi-Agent LLM Framework for Rating the Quality of Surgical Feedback [52.053200111780626]
  Verbal feedback delivered by attending surgeons in the operating room plays a critical formative role in resident skill acquisition.<n>We introduce a two-stage LLM-based framework that discovers interpretable feedback quality criteria grounded in the context of surgical training.<n>Our method uses multi-agent and surgical domain knowledge injection to discover a small set of human interpretable scoring criteria.
  arXiv  Detail & Related papers  (2026-05-25T05:31:44Z)
• Measuring What Matters!! Assessing Therapeutic Principles in Mental-Health Conversation [11.873690830791022]
  We study the problem of evaluating AI-generated therapist-like responses for clinically grounded appropriateness and effectiveness.<n>We introduce FAITH-M, a benchmark annotated with expert-assigned ordinal ratings.<n>We propose CARE, a multi-stage evaluation framework that integrates intra-dialogue context, contrastive retrieval, and knowledge-distilled chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-04-07T12:30:54Z)
• Benchmarking Deep Learning-Based Reconstruction Methods for Photoacoustic Computed Tomography with Clinically Relevant Synthetic Datasets [5.916152021724446]
  Deep learning (DL)-based image reconstruction methods for photoacoustic computed tomography (PACT) have developed rapidly in recent years.<n>Most existing methods have not employed standardized datasets, and their evaluations rely on traditional image quality (IQ) metrics that may lack clinical relevance.<n>A benchmarking framework is proposed that provides open-source, anatomically plausible synthetic datasets and evaluation strategies for DL-based acoustic inversion methods in PACT.
  arXiv  Detail & Related papers  (2026-01-23T20:52:35Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• STROKEVISION-BENCH: A Multimodal Video And 2D Pose Benchmark For Tracking Stroke Recovery [41.140934816875806]
  We introduce StrokeVision-Bench, the first-ever dedicated dataset of stroke patients performing clinically structured block transfer tasks.<n>StrokeVision-Bench comprises 1,000 annotated videos categorized into four clinically meaningful action classes.<n>We benchmark several state-of-the-art video action recognition and skeleton-based action classification methods to establish performance baselines.
  arXiv  Detail & Related papers  (2025-09-02T18:48:37Z)
• GEMeX-RMCoT: An Enhanced Med-VQA Dataset for Region-Aware Multimodal Chain-of-Thought Reasoning [60.03671205298294]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability.<n>This work first proposes a Region-Aware Multimodal Chain-of-Thought dataset, in which the process of producing an answer is preceded by a sequence of intermediate reasoning steps.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitation [0.30693357740321775]
  This paper introduces a new graph-based model for assessing rehabilitation exercises.<n>Dense connections and GRU mechanisms are used to rapidly process large 3D skeleton inputs.<n>The evaluation of our proposed approach on the KIMORE and UI-PRMD datasets highlighted its potential.
  arXiv  Detail & Related papers  (2023-12-21T00:38:31Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• A Review of Computational Approaches for Evaluation of Rehabilitation Exercises [58.720142291102135]
  This paper reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches.
  arXiv  Detail & Related papers  (2020-02-29T22:18:56Z)
• Opportunities of a Machine Learning-based Decision Support System for Stroke Rehabilitation Assessment [64.52563354823711]
  Rehabilitation assessment is critical to determine an adequate intervention for a patient. Current practices of assessment mainly rely on therapist's experience, and assessment is infrequently executed due to the limited availability of a therapist. We developed an intelligent decision support system that can identify salient features of assessment using reinforcement learning.
  arXiv  Detail & Related papers  (2020-02-27T17:04:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.