A Multi-body Tracking Framework -- From Rigid Objects to Kinematic Structures

• URL: http://arxiv.org/abs/2208.01502v1
• Date: Tue, 2 Aug 2022 14:49:34 GMT
• Title: A Multi-body Tracking Framework -- From Rigid Objects to Kinematic Structures
• Authors: Manuel Stoiber, Martin Sundermeyer, Wout Boerdijk, Rudolph Triebel
• Abstract summary: Most model-based 3D tracking methods only consider rigid objects. We propose a flexible framework that allows the extension of existing 6DoF algorithms to kinematic structures. Based on the proposed framework, we extend ICG, which is a state-of-the-art rigid object tracking algorithm, to multi-body tracking.
• Score: 23.933556023366695
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kinematic structures are very common in the real world. They range from simple articulated objects to complex mechanical systems. However, despite their relevance, most model-based 3D tracking methods only consider rigid objects. To overcome this limitation, we propose a flexible framework that allows the extension of existing 6DoF algorithms to kinematic structures. Our approach focuses on methods that employ Newton-like optimization techniques, which are widely used in object tracking. The framework considers both tree-like and closed kinematic structures and allows a flexible configuration of joints and constraints. To project equations from individual rigid bodies to a multi-body system, Jacobians are used. For closed kinematic chains, a novel formulation that features Lagrange multipliers is developed. In a detailed mathematical proof, we show that our constraint formulation leads to an exact kinematic solution and converges in a single iteration. Based on the proposed framework, we extend ICG, which is a state-of-the-art rigid object tracking algorithm, to multi-body tracking. For the evaluation, we create a highly-realistic synthetic dataset that features a large number of sequences and various robots. Based on this dataset, we conduct a wide variety of experiments that demonstrate the excellent performance of the developed framework and our multi-body tracker.

Related papers

• Hierarchical Contact-Rich Trajectory Optimization for Multi-Modal Manipulation using Tight Convex Relaxations [12.578064173652148]
  We present a novel framework for designing trajectories of robots, objects, and contacts efficiently for contact-rich manipulation. We propose a hierarchical optimization framework where Mixed-Integer Linear Program (MILP) selects optimal contacts between robot & object. We present a convex relaxation of bilinear constraints using binary encoding technique such that MILP can provide tighter solutions.
  arXiv  Detail & Related papers  (2025-03-11T01:40:23Z)
• Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering [2.763111962660262]
  We study the problem of self-supervised 3D scene flow estimation from real large-scale raw point cloud sequences. We propose a novel clustering approach that allows for combination of overlapping soft clusters as well as non-overlapping rigid clusters. Our method especially excels in resolving flow in complicated dynamic scenes with multiple independently moving objects close to each other.
  arXiv  Detail & Related papers  (2024-04-12T10:04:03Z)
• Weak-to-Strong 3D Object Detection with X-Ray Distillation [75.47580744933724]
  We propose a versatile technique that seamlessly integrates into any existing framework for 3D Object Detection. X-Ray Distillation with Object-Complete Frames is suitable for both supervised and semi-supervised settings. Our proposed methods surpass state-of-the-art in semi-supervised learning by 1-1.5 mAP.
  arXiv  Detail & Related papers  (2024-03-31T13:09:06Z)
• Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking [61.69892497726235]
  Composite Node Message Passing Network (CoNo-Link) is a framework for modeling ultra-long frames information for association. In addition to the previous method of treating objects as nodes, the network innovatively treats object trajectories as nodes for information interaction. Our model can learn better predictions on longer-time scales by adding composite nodes.
  arXiv  Detail & Related papers  (2023-12-14T14:00:30Z)
• Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs [53.66070434419739]
  Generalizable articulated object manipulation is essential for home-assistant robots. We propose a kinematic-aware prompting framework that prompts Large Language Models with kinematic knowledge of objects to generate low-level motion waypoints. Our framework outperforms traditional methods on 8 categories seen and shows a powerful zero-shot capability for 8 unseen articulated object categories.
  arXiv  Detail & Related papers  (2023-11-06T03:26:41Z)
• Differentiable Simulation of Soft Multi-body Systems [99.4302215142673]
  We develop a top-down matrix assembly algorithm within Projective Dynamics. We derive a differentiable control framework for soft articulated bodies driven by muscles, joint torques, or pneumatic tubes.
  arXiv  Detail & Related papers  (2022-05-03T20:03:22Z)
• A general framework for modeling and dynamic simulation of multibody systems using factor graphs [0.8701566919381223]
  We present a novel general framework grounded in the factor graph theory to solve kinematic and dynamic problems for multi-body systems. We describe how to build factor graphs to model and simulate multibody systems using both, independent and dependent coordinates. The proposed framework has been tested in extensive simulations and validated against a commercial multibody software.
  arXiv  Detail & Related papers  (2021-01-08T06:45:45Z)
• Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation [89.82169646672872]
  We propose an unsupervised vision-based system to estimate the joint configurations of the robot arm from a sequence of RGB or RGB-D images without knowing the model a priori. We combine a classical geometric formulation with deep learning and extend the use of epipolar multi-rigid-body constraints to solve this task.
  arXiv  Detail & Related papers  (2020-11-30T20:46:48Z)
• Scalable Differentiable Physics for Learning and Control [99.4302215142673]
  Differentiable physics is a powerful approach to learning and control problems that involve physical objects and environments. We develop a scalable framework for differentiable physics that can support a large number of objects and their interactions.
  arXiv  Detail & Related papers  (2020-07-04T19:07:51Z)
• Learning Physical Constraints with Neural Projections [16.09436906471513]
  We propose a new family of neural networks to predict the behaviors of physical systems by learning their underpinning constraints. A neural projection operator lies at the heart of our approach, composed of a lightweight network with an embedded recursion architecture. We demonstrated the efficacy of our approach by learning a set of challenging physical systems all in a unified and simple fashion.
  arXiv  Detail & Related papers  (2020-06-23T04:19:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.