Intelligent Knee Sleeves: A Real-time Multimodal Dataset for 3D Lower Body Motion Estimation Using Smart Textile

• URL: http://arxiv.org/abs/2311.12829v1
• Date: Mon, 2 Oct 2023 00:34:21 GMT
• Title: Intelligent Knee Sleeves: A Real-time Multimodal Dataset for 3D Lower Body Motion Estimation Using Smart Textile
• Authors: Wenwen Zhang, Arvin Tashakori, Zenan Jiang, Amir Servati, Harishkumar Narayana, Saeid Soltanian, Rou Yi Yeap, Meng Han Ma, Lauren Toy, Peyman Servati
• Abstract summary: We present a multimodal dataset with benchmarks collected using a novel pair of Intelligent Knee Sleeves for human pose estimation. Our system utilizes synchronized datasets that comprise time-series data from the Knee Sleeves and the corresponding ground truth labels from the visualized motion capture camera system. We employ these to generate 3D human models solely based on the wearable data of individuals performing different activities.
• Score: 2.2008680042670123
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The kinematics of human movements and locomotion are closely linked to the activation and contractions of muscles. To investigate this, we present a multimodal dataset with benchmarks collected using a novel pair of Intelligent Knee Sleeves (Texavie MarsWear Knee Sleeves) for human pose estimation. Our system utilizes synchronized datasets that comprise time-series data from the Knee Sleeves and the corresponding ground truth labels from the visualized motion capture camera system. We employ these to generate 3D human models solely based on the wearable data of individuals performing different activities. We demonstrate the effectiveness of this camera-free system and machine learning algorithms in the assessment of various movements and exercises, including extension to unseen exercises and individuals. The results show an average error of 7.21 degrees across all eight lower body joints when compared to the ground truth, indicating the effectiveness and reliability of the Knee Sleeve system for the prediction of different lower body joints beyond the knees. The results enable human pose estimation in a seamless manner without being limited by visual occlusion or the field of view of cameras. Our results show the potential of multimodal wearable sensing in a variety of applications from home fitness to sports, healthcare, and physical rehabilitation focusing on pose and movement estimation.

Related papers

• Continual Imitation Learning for Prosthetic Limbs [0.7922558880545526]
  Motorized bionic limbs offer promise, but their utility depends on mimicking the evolving synergy of human movement in various settings. We present a novel model for bionic prostheses' application that leverages camera-based motion capture and wearable sensor data. We propose a model that can multitask, adapt continually, anticipate movements, and refine locomotion.
  arXiv  Detail & Related papers  (2024-05-02T09:22:54Z)
• 3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data [4.130944152992895]
  We propose a novel biomechanics-aware network that directly outputs 3D kinematics from two input views. Our experiments demonstrate that the proposed approach, only trained on synthetic data, outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-20T17:33:40Z)
• Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement [65.08165593201437]
  We explore egocentric whole-body motion capture using a single fisheye camera, which simultaneously estimates human body and hand motion. This task presents significant challenges due to the lack of high-quality datasets, fisheye camera distortion, and human body self-occlusion. We propose a novel approach that leverages FisheyeViT to extract fisheye image features, which are converted into pixel-aligned 3D heatmap representations for 3D human body pose prediction.
  arXiv  Detail & Related papers  (2023-11-28T07:13:47Z)
• Muscles in Action [22.482090207522358]
  We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. We learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.
  arXiv  Detail & Related papers  (2022-12-05T16:47:09Z)
• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars [80.05743236282564]
  Real-time tracking of human body motion is crucial for immersive experiences in AR/VR. We present a reinforcement learning framework that takes in sparse signals from an HMD and two controllers. We show that a single policy can be robust to diverse locomotion styles, different body sizes, and novel environments.
  arXiv  Detail & Related papers  (2022-09-20T00:25:54Z)
• Occlusion-Robust Multi-Sensory Posture Estimation in Physical Human-Robot Interaction [10.063075560468798]
  2D postures from OpenPose over a single camera, and the trajectory of the interacting robot while the human performs a task. We use 2D postures from OpenPose over a single camera, and the trajectory of the interacting robot while the human performs a task. We show that our multi-sensory system resolves human kinematic redundancy better than posture estimation solely using OpenPose or posture estimation solely using the robot's trajectory.
  arXiv  Detail & Related papers  (2022-08-12T20:41:09Z)
• Imposing Temporal Consistency on Deep Monocular Body Shape and Pose Estimation [67.23327074124855]
  This paper presents an elegant solution for the integration of temporal constraints in the fitting process. We derive parameters of a sequence of body models, representing shape and motion of a person, including jaw poses, facial expressions, and finger poses. Our approach enables the derivation of realistic 3D body models from image sequences, including facial expression and articulated hands.
  arXiv  Detail & Related papers  (2022-02-07T11:11:55Z)
• LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies [78.17425779503047]
  We propose a novel neural implicit representation for the human body. It is fully differentiable and optimizable with disentangled shape and pose latent spaces. Our model can be trained and fine-tuned directly on non-watertight raw data with well-designed losses.
  arXiv  Detail & Related papers  (2021-11-30T04:10:57Z)
• 3D Human Pose Estimation for Free-form Activity Using WiFi Signals [5.2245900672091]
  Winect is a 3D human pose tracking system for free-form activity using commodity WiFi devices. Our system tracks free-form activity by estimating a 3D skeleton pose that consists of a set of joints of the human body.
  arXiv  Detail & Related papers  (2021-10-15T18:47:16Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.