ForcePose: A Deep Learning Approach for Force Calculation Based on Action Recognition Using MediaPipe Pose Estimation Combined with Object Detection

• URL: http://arxiv.org/abs/2503.22363v1
• Date: Fri, 28 Mar 2025 12:13:56 GMT
• Title: ForcePose: A Deep Learning Approach for Force Calculation Based on Action Recognition Using MediaPipe Pose Estimation Combined with Object Detection
• Abstract summary: ForcePose is a novel deep learning framework that estimates applied forces by combining human pose estimation with object detection.<n>We've developed a specialized neural network that processes both spatial and temporal features to predict force magnitude and direction without needing any physical sensors.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Force estimation in human-object interactions is crucial for various fields like ergonomics, physical therapy, and sports science. Traditional methods depend on specialized equipment such as force plates and sensors, which makes accurate assessments both expensive and restricted to laboratory settings. In this paper, we introduce ForcePose, a novel deep learning framework that estimates applied forces by combining human pose estimation with object detection. Our approach leverages MediaPipe for skeletal tracking and SSD MobileNet for object recognition to create a unified representation of human-object interaction. We've developed a specialized neural network that processes both spatial and temporal features to predict force magnitude and direction without needing any physical sensors. After training on our dataset of 850 annotated videos with corresponding force measurements, our model achieves a mean absolute error of 5.83 N in force magnitude and 7.4 degrees in force direction. When compared to existing computer vision approaches, our method performs 27.5% better while still offering real-time performance on standard computing hardware. ForcePose opens up new possibilities for force analysis in diverse real-world scenarios where traditional measurement tools are impractical or intrusive. This paper discusses our methodology, the dataset creation process, evaluation metrics, and potential applications across rehabilitation, ergonomics assessment, and athletic performance analysis.

Related papers

• HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction [48.4618478154232]
  HiPHI is a 600+ hour scale high-fidelity whole-body human motion dataset.<n>It provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories.<n>We introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications.
  arXiv  Detail & Related papers  (2026-08-17T07:54:26Z)
• Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues? [75.43540911934191]
  Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations.<n>We propose a new learning paradigm that combines hand-object masked training and an HOI-dynamics-aware decoder.<n>We show that our training strategy exploits hand- and object-centric information more effectively than existing models.
  arXiv  Detail & Related papers  (2026-07-09T14:09:29Z)
• ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations [24.273078797508884]
  ThorArena is a benchmark for evaluating force-aware humanoid interaction based on human demonstrations with synchronized motion and force measurements.<n>We propose force-aware evaluation metrics that jointly assess whole-body tracking accuracy, robustness under different force levels, control effort, and episode survival.
  arXiv  Detail & Related papers  (2026-07-07T09:26:51Z)
• Adaptive vs. Static Robot-to-Human Handover: A Study on Orientation and Approach Direction [0.3425341633647625]
  This work presents a novel adaptive framework that adjusts dynamically the object's delivery pose in real time based on the user's hand pose and the intended downstream task.<n>A comprehensive user study compares the proposed adaptive approach against a static baseline across multiple tasks.<n>Results demonstrate that dynamic alignment significantly reduces users' cognitive workload and physiological stress, while increasing perceived trust in the robot's reliability.
  arXiv  Detail & Related papers  (2026-04-24T09:14:44Z)
• PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models [38.21000830724267]
  We propose PhysQuantAgent, a framework for real-world object mass estimation using Vision-Language Models (VLMs)<n>We introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object.<n> Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
  arXiv  Detail & Related papers  (2026-03-17T02:35:36Z)
• PP-Motion: Physical-Perceptual Fidelity Evaluation for Human Motion Generation [84.40992836326613]
  Human motion generation has found widespread applications in AR/VR, film, sports, and medical rehabilitation.<n>Previous approaches have attempted at motion fidelity evaluation using human perception or physical constraints.<n>We propose PP-Motion, a novel data-driven metric to evaluate both physical and perceptual fidelity of human motion.
  arXiv  Detail & Related papers  (2025-08-11T16:59:15Z)
• Physics-informed Ground Reaction Dynamics from Human Motion Capture [4.4795626402834055]
  We propose a novel method for estimating human ground reaction dynamics directly from motion capture data.<n>We introduce a highly accurate and robust method for computing ground reaction forces from motion capture data using Euler's integration scheme and PD algorithm.<n>The proposed approach was tested on the GroundLink dataset.
  arXiv  Detail & Related papers  (2025-07-02T04:02:16Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• Physics Inspired Hybrid Attention for SAR Target Recognition [61.01086031364307]
  We propose a physics inspired hybrid attention (PIHA) mechanism and the once-for-all (OFA) evaluation protocol to address the issues. PIHA leverages the high-level semantics of physical information to activate and guide the feature group aware of local semantics of target. Our method outperforms other state-of-the-art approaches in 12 test scenarios with same ASC parameters.
  arXiv  Detail & Related papers  (2023-09-27T14:39:41Z)
• Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions [82.90906153293585]
  We propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task. We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects.
  arXiv  Detail & Related papers  (2022-06-25T09:55:39Z)
• Learning suction graspability considering grasp quality and robot reachability for bin-picking [4.317666242093779]
  We propose an intuitive geometric analytic-based grasp quality evaluation metric. We further incorporate a reachability evaluation metric. Experiment results show that our intuitive grasp quality evaluation metric is competitive with a physically-inspired metric.
  arXiv  Detail & Related papers  (2021-11-04T00:55:42Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State [25.121899443298567]
  Vision-based deep learning using convolutional neural networks is a promising approach for providing useful force estimates. We present a force estimation neural network that uses RGB images and robot state as inputs. It showed comparable accuracy but faster computation times than a baseline recurrent neural network, making it better suited for real-time applications.
  arXiv  Detail & Related papers  (2020-11-04T04:00:07Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)
• Gaining a Sense of Touch. Physical Parameters Estimation using a Soft Gripper and Neural Networks [3.0892724364965005]
  There is not enough research on physical parameters estimation using deep learning algorithms on measurements from direct interaction with objects using robotic grippers. We propose a trainable system for the regression of a stiffness coefficient and provided extensive experiments using the physics simulator environment. Our system can reliably estimate the stiffness of an object using the Yale OpenHand soft gripper based on readings from Inertial Measurement Units (IMUs) attached to its fingers.
  arXiv  Detail & Related papers  (2020-03-02T11:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.