Vision6D: 3D-to-2D Interactive Visualization and Annotation Tool for 6D Pose Estimation

• URL: http://arxiv.org/abs/2504.15329v1
• Date: Mon, 21 Apr 2025 16:36:52 GMT
• Title: Vision6D: 3D-to-2D Interactive Visualization and Annotation Tool for 6D Pose Estimation
• Authors: Yike Zhang, Eduardo Davalos, Jack Noble,
• Abstract summary: This paper presents an interactive 3D-to-2D visualization and annotation tool to support the 6D pose estimation research community.<n>Vision6D generates accurate pose annotations via visual cues in an intuitive 3D user interface.
• Score: 4.250558597144547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate 6D pose estimation has gained more attention over the years for robotics-assisted tasks that require precise interaction with physical objects. This paper presents an interactive 3D-to-2D visualization and annotation tool to support the 6D pose estimation research community. To the best of our knowledge, the proposed work is the first tool that allows users to visualize and manipulate 3D objects interactively on a 2D real-world scene, along with a comprehensive user study. This system supports robust 6D camera pose annotation by providing both visual cues and spatial relationships to determine object position and orientation in various environments. The annotation feature in Vision6D is particularly helpful in scenarios where the transformation matrix between the camera and world objects is unknown, as it enables accurate annotation of these objects' poses using only the camera intrinsic matrix. This capability serves as a foundational step in developing and training advanced pose estimation models across various domains. We evaluate Vision6D's effectiveness by utilizing widely-used open-source pose estimation datasets Linemod and HANDAL through comparisons between the default ground-truth camera poses with manual annotations. A user study was performed to show that Vision6D generates accurate pose annotations via visual cues in an intuitive 3D user interface. This approach aims to bridge the gap between 2D scene projections and 3D scenes, offering an effective way for researchers and developers to solve 6D pose annotation related problems. The software is open-source and publicly available at https://github.com/InteractiveGL/vision6D.

Related papers

• Any6D: Model-free 6D Pose Estimation of Novel Objects [76.30057578269668]
  We introduce Any6D, a model-free framework for 6D object pose estimation.<n>It requires only a single RGB-D anchor image to estimate both the 6D pose and size of unknown objects in novel scenes.<n>We evaluate our method on five challenging datasets.
  arXiv  Detail & Related papers  (2025-03-24T13:46:21Z)
• 6D Object Pose Tracking in Internet Videos for Robotic Manipulation [20.22297850525832]
  We develop a new method that estimates the 6D pose of any object in the input image without prior knowledge of the object itself.<n>We extract smooth 6D object trajectories from Internet videos by carefully tracking the detected objects across video frames.<n>We demonstrate significant improvements over existing state-of-the-art RGB 6D pose estimation methods.
  arXiv  Detail & Related papers  (2025-03-13T12:33:34Z)
• Self-learning Canonical Space for Multi-view 3D Human Pose Estimation [57.969696744428475]
  Multi-view 3D human pose estimation is naturally superior to single view one. The accurate annotation of these information is hard to obtain. We propose a fully self-supervised framework, named cascaded multi-view aggregating network (CMANet) CMANet is superior to state-of-the-art methods in extensive quantitative and qualitative analysis.
  arXiv  Detail & Related papers  (2024-03-19T04:54:59Z)
• Weakly-supervised Pre-training for 3D Human Pose Estimation via Perspective Knowledge [36.65402869749077]
  We propose a novel method to extract weak 3D information directly from 2D images without 3D pose supervision. We propose a weakly-supervised pre-training (WSP) strategy to distinguish the depth relationship between two points in an image. WSP achieves state-of-the-art results on two widely-used benchmarks.
  arXiv  Detail & Related papers  (2022-11-22T03:35:15Z)
• Imitrob: Imitation Learning Dataset for Training and Evaluating 6D Object Pose Estimators [20.611000416051546]
  This paper introduces a dataset for training and evaluating methods for 6D pose estimation of hand-held tools in task demonstrations captured by a standard RGB camera. The dataset contains image sequences of nine different tools and twelve manipulation tasks with two camera viewpoints, four human subjects, and left/right hand.
  arXiv  Detail & Related papers  (2022-09-16T14:43:46Z)
• Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [64.7198752089041]
  Given a set of known 3D objects and an RGB or RGB-D input image, we detect and estimate the 6D pose of each object. Our approach iteratively refines both pose and correspondence in a tightly coupled manner, allowing us to dynamically remove outliers to improve accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:00:08Z)
• Weakly Supervised Learning of Keypoints for 6D Object Pose Estimation [73.40404343241782]
  We propose a weakly supervised 6D object pose estimation approach based on 2D keypoint detection. Our approach achieves comparable performance with state-of-the-art fully supervised approaches.
  arXiv  Detail & Related papers  (2022-03-07T16:23:47Z)
• Unsupervised Learning of Visual 3D Keypoints for Control [104.92063943162896]
  Learning sensorimotor control policies from high-dimensional images crucially relies on the quality of the underlying visual representations. We propose a framework to learn such a 3D geometric structure directly from images in an end-to-end unsupervised manner. These discovered 3D keypoints tend to meaningfully capture robot joints as well as object movements in a consistent manner across both time and 3D space.
  arXiv  Detail & Related papers  (2021-06-14T17:59:59Z)
• H2O: Two Hands Manipulating Objects for First Person Interaction Recognition [70.46638409156772]
  We present a comprehensive framework for egocentric interaction recognition using markerless 3D annotations of two hands manipulating objects. Our method produces annotations of the 3D pose of two hands and the 6D pose of the manipulated objects, along with their interaction labels for each frame. Our dataset, called H2O (2 Hands and Objects), provides synchronized multi-view RGB-D images, interaction labels, object classes, ground-truth 3D poses for left & right hands, 6D object poses, ground-truth camera poses, object meshes and scene point clouds.
  arXiv  Detail & Related papers  (2021-04-22T17:10:42Z)
• 3D Object Recognition By Corresponding and Quantizing Neural 3D Scene Representations [29.61554189447989]
  We propose a system that learns to detect objects and infer their 3D poses in RGB-D images. Many existing systems can identify objects and infer 3D poses, but they heavily rely on human labels and 3D annotations.
  arXiv  Detail & Related papers  (2020-10-30T13:56:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.