SDF-based RGB-D Camera Tracking in Neural Scene Representations

• URL: http://arxiv.org/abs/2205.02079v1
• Date: Wed, 4 May 2022 14:18:39 GMT
• Title: SDF-based RGB-D Camera Tracking in Neural Scene Representations
• Authors: Leonard Bruns, Fereidoon Zangeneh, Patric Jensfelt
• Abstract summary: We consider the problem of tracking the 6D pose of a moving RGB-D camera in a neural scene representation. In particular, we propose to track an RGB-D camera using a signed distance field-based representation and show that compared to density-based representations, tracking can be sped up.
• Score: 4.83420384410068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of tracking the 6D pose of a moving RGB-D camera in a neural scene representation. Different such representations have recently emerged, and we investigate the suitability of them for the task of camera tracking. In particular, we propose to track an RGB-D camera using a signed distance field-based representation and show that compared to density-based representations, tracking can be sped up, which enables more robust and accurate pose estimates when computation time is limited.

Related papers

• Depth Attention for Robust RGB Tracking [21.897255266278275]
  We propose a new framework that leverages monocular depth estimation to counter the challenges of tracking targets that are out of view or affected by motion blur in RGB video sequences. To the best of our knowledge, we are the first to propose a depth attention mechanism and to formulate a simple framework that allows seamlessly integration of depth information with state of the art tracking algorithms.
  arXiv  Detail & Related papers  (2024-10-27T09:47:47Z)
• CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Cameras [43.699819213559515]
  Existing datasets for RGB-DVS tracking are collected with DVS346 camera and their resolution ($346 times 260$) is low for practical applications. We build the first unaligned frame-event dataset CRSOT collected with a specially built data acquisition system. We propose a novel unaligned object tracking framework that can realize robust tracking even using the loosely aligned RGB-Event data.
  arXiv  Detail & Related papers  (2024-01-05T14:20:22Z)
• RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [72.13154206106259]
  We propose a novel pipeline that decouples the 6D pose and size estimation to mitigate the influence of imperfect scales on rigid transformations. Specifically, we leverage a pre-trained monocular estimator to extract local geometric information. A separate branch is designed to directly recover the metric scale of the object based on category-level statistics.
  arXiv  Detail & Related papers  (2023-09-19T02:20:26Z)
• ARKitTrack: A New Diverse Dataset for Tracking Using Mobile RGB-D Data [75.73063721067608]
  We propose a new RGB-D tracking dataset for both static and dynamic scenes captured by consumer-grade LiDAR scanners equipped on Apple's iPhone and iPad. ARKitTrack contains 300 RGB-D sequences, 455 targets, and 229.7K video frames in total. In-depth empirical analysis has verified that the ARKitTrack dataset can significantly facilitate RGB-D tracking and that the proposed baseline method compares favorably against the state of the arts.
  arXiv  Detail & Related papers  (2023-03-24T09:51:13Z)
• Virtually increasing the measurement frequency of LIDAR sensor utilizing a single RGB camera [1.3706331473063877]
  This research suggests using a mono camera to virtually enhance the frame rate of LIDARs. We achieve state-of-the-art performance on large public datasets in terms of accuracy and similarity to real measurements.
  arXiv  Detail & Related papers  (2023-02-10T11:43:35Z)
• Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography [54.36608424943729]
  We show that in a ''long-burst'', forty-two 12-megapixel RAW frames captured in a two-second sequence, there is enough parallax information from natural hand tremor alone to recover high-quality scene depth. We devise a test-time optimization approach that fits a neural RGB-D representation to long-burst data and simultaneously estimates scene depth and camera motion.
  arXiv  Detail & Related papers  (2022-12-22T18:54:34Z)
• RGBD Object Tracking: An In-depth Review [89.96221353160831]
  We firstly review RGBD object trackers from different perspectives, including RGBD fusion, depth usage, and tracking framework. We benchmark a representative set of RGBD trackers, and give detailed analyses based on their performances.
  arXiv  Detail & Related papers  (2022-03-26T18:53:51Z)
• Visual Object Tracking on Multi-modal RGB-D Videos: A Review [16.098468526632473]
  The goal of this review is to summarize the relative knowledge of the research filed of RGB-D tracking. To be specific, we will generalize the related RGB-D tracking benchmarking datasets as well as the corresponding performance measurements.
  arXiv  Detail & Related papers  (2022-01-23T08:02:49Z)
• RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video [76.86512780916827]
  We present the first real-time method for motion capture of skeletal pose and 3D surface geometry of hands from a single RGB camera. In order to address the inherent depth ambiguities in RGB data, we propose a novel multi-task CNN. We experimentally verify the individual components of our RGB two-hand tracking and 3D reconstruction pipeline.
  arXiv  Detail & Related papers  (2021-06-22T12:53:56Z)
• Real-time RGBD-based Extended Body Pose Estimation [57.61868412206493]
  We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation. We train estimators of body pose and facial expression parameters.
  arXiv  Detail & Related papers  (2021-03-05T13:37:50Z)
• RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking [16.06615504110132]
  We propose to use an event-based camera to increase the speed of 3D object tracking in 6 degrees of freedom. This application requires handling very high object speed to convey compelling AR experiences. We develop a deep learning approach, which combines an existing RGB-D network along with a novel event-based network in a cascade fashion.
  arXiv  Detail & Related papers  (2020-06-09T01:55:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.