Kinematic Knowledge Maps for Pattern Alignment: Structured Latent Representational Learning in Multimodal Gait Analysis

• URL: http://arxiv.org/abs/2608.20969v1
• Date: Fri, 21 Aug 2026 10:46:40 GMT
• Title: Kinematic Knowledge Maps for Pattern Alignment: Structured Latent Representational Learning in Multimodal Gait Analysis
• Abstract summary: We present ScoliDetect, an explainable framework for adolescent idiopathic scoliosis screening from monocular gait video.<n>KKM is a fixed-index structured representation that encodes gait features across absolute motion.<n>KKM provides inherent, factor-level attributions mapped directly to specific kinematic phases and skeletal indices.
• Score: 1.7030020917166933
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal clinical AI is limited by weakly aligned inputs and the absence of domain-specific interpretable representations, particularly when learning from dense video stream, structured time-series, and template-based kinematic text. Here we present ScoliDetect, an explainable framework for adolescent idiopathic scoliosis screening from monocular gait video, built around a kinematic knowledge map (KKM) and complementary template-based kinematic text derived from per-sequence pose statics. KKM is a fixed-index structured representation that encodes gait features across absolute motion, self-skeleton configuration and joint-joint signal correlation, providing anchor-referenced multimodal fusion and factor-level interpretation. We integrate video, KKM, and template-based kinematic text through bidirectional cross-attention with latent-bottleneck aggregation. In a multicenter cohort (n = 1,858 after exclusions), prespecified supervised ablations on an external screening cohort show that KKM-mediated multimodal fusion outperforms unimodal models and late concatenation. Under a staged training protocol, trimodal contrastive pretraining is applied after architecture selection as representation initialization, improving external ROC-AUC from 0.961 to 0.972. Furthermore, the structured nature of the KKM provides inherent, factor-level attributions mapped directly to specific kinematic phases and skeletal indices, offering verifiable interpretability. The results demonstrate that embedding explicit structural topologies into latent spaces significantly enhances both the generalization and explainability of multimodal pattern analysis systems.

Related papers

• PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images [51.428093790826814]
  We propose pathology Autorgressive modeling (PathAR) to factorize structure and appearance for modality-label-conditioned pathology generation.<n>PathAR stabilizes morphology under heterogeneous modality-specific appearances and enables spatially aligned image-mask pair generation.
  arXiv  Detail & Related papers  (2026-06-01T01:43:48Z)
• Semantic Generative Tuning for Unified Multimodal Models [62.18894352635965]
  Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture.<n> prevailing training paradigms independently optimize understanding via sparse text signals and generation through dense pixel objectives.<n>This work presents the first systematic investigation into generative post-training, where we formulate hierarchical visual tasks as generative proxies to bridge the isolation in UMMs.
  arXiv  Detail & Related papers  (2026-05-18T17:46:46Z)
• Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs [84.3271821505699]
  Chain of Modality (CoM) is an agentic framework that transitions multimodal fusion from passive concatenation to dynamic orchestration.<n> operating in either a training-free or a data-efficient SFT setting, CoM achieves robust and consistent generalization across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T01:21:14Z)
• Multimodal Structure Learning: Disentangling Shared and Specific Topology via Cross-Modal Graphical Lasso [12.238577096609761]
  We propose Cross-Modal Graphical Lasso (CM-GLasso) for learning interpretable multimodal representations.<n>By coupling a novel text-visualization strategy with a unified vision-language encoder, we strictly align multimodal features into a shared latent space.<n>We show that CM-GLasso establishes a new state-of-the-art in generative classification and dense semantic segmentation tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:07:59Z)
• Focus-to-Perceive Representation Learning: A Cognition-Inspired Hierarchical Framework for Endoscopic Video Analysis [18.349979396713646]
  We propose Focus-to-Perceive Learning (FPRL), a cognition-inspired hierarchical framework that emulates clinical examination.<n>FPRL first focuses on intra-frame lesion-centric regions to learn static semantics, and then perceives their evolution across frames to model contextual semantics.<n>Experiments on 11 endoscopic video datasets show that FPRL achieves superior performance across diverse downstream tasks.
  arXiv  Detail & Related papers  (2026-03-26T14:06:48Z)
• CUS-GS: A Compact Unified Structured Gaussian Splatting Framework for Multimodal Scene Representation [16.85102888388904]
  CUS-GS is a compact unified structured Gaussian Splatting representation.<n>We propose a feature-aware significance evaluation strategy to guide anchor growing and pruning.<n>CUS-GS achieves competitive performance compared to state-of-the-art methods using as few as 6M parameters.
  arXiv  Detail & Related papers  (2025-11-22T03:42:49Z)
• MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition [49.91188543847175]
  Multi-Skeleton Contrastive Learning (MS-CLR) is a framework that aligns pose representations across multiple skeleton conventions extracted from the same sequence.<n>MS-CLR consistently improves performance over strong single-skeleton contrastive learning baselines.<n>A multi-skeleton ensemble further boosts performance, setting new state-of-the-art results on both datasets.
  arXiv  Detail & Related papers  (2025-08-20T17:58:03Z)
• DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detection [7.117824587276951]
  This study offers a dual-path architecture called the Dual-Branch Adaptive Multiscale Stemporal Framework (DAMS), which is based on multilevel feature and decoupling fusion.<n>The main processing path integrates the Adaptive Multiscale Time Pyramid Network (AMTPN) with the Convolutional Block Attention Mechanism (CBAM)
  arXiv  Detail & Related papers  (2025-07-28T08:42:00Z)
• Enhanced Partially Relevant Video Retrieval through Inter- and Intra-Sample Analysis with Coherence Prediction [18.24629930062925]
  Partially Relevant Video Retrieval aims to retrieve the target video that is partially relevant to a text query.<n>Existing methods coarsely align paired videos and text queries to construct the semantic space.<n>We propose a novel PRVR framework to systematically exploit inter-sample correlation and intra-sample redundancy.
  arXiv  Detail & Related papers  (2025-04-28T09:52:46Z)
• Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation [48.267001230607306]
  We propose a pretraining framework that enhances multiscale consistency in EM volumes. Our approach leverages a Siamese network architecture, integrating strong and weak data augmentations. It effectively captures voxel and feature consistency, showing promise for learning transferable representations for EM analysis.
  arXiv  Detail & Related papers  (2023-08-19T05:49:13Z)
• Multi-View Clustering via Semi-non-negative Tensor Factorization [120.87318230985653]
  We develop a novel multi-view clustering based on semi-non-negative tensor factorization (Semi-NTF) Our model directly considers the between-view relationship and exploits the between-view complementary information. In addition, we provide an optimization algorithm for the proposed method and prove mathematically that the algorithm always converges to the stationary KKT point.
  arXiv  Detail & Related papers  (2023-03-29T14:54:19Z)
• Support-set based Multi-modal Representation Enhancement for Video Captioning [121.70886789958799]
  We propose a Support-set based Multi-modal Representation Enhancement (SMRE) model to mine rich information in a semantic subspace shared between samples. Specifically, we propose a Support-set Construction (SC) module to construct a support-set to learn underlying connections between samples and obtain semantic-related visual elements. During this process, we design a Semantic Space Transformation (SST) module to constrain relative distance and administrate multi-modal interactions in a self-supervised way.
  arXiv  Detail & Related papers  (2022-05-19T03:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.