Deep Soft Procrustes for Markerless Volumetric Sensor Alignment

• URL: http://arxiv.org/abs/2003.10176v1
• Date: Mon, 23 Mar 2020 10:51:32 GMT
• Title: Deep Soft Procrustes for Markerless Volumetric Sensor Alignment
• Authors: Vladimiros Sterzentsenko and Alexandros Doumanoglou and Spyridon Thermos and Nikolaos Zioulis and Dimitrios Zarpalas and Petros Daras
• Abstract summary: In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
• Score: 81.13055566952221
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advent of consumer grade depth sensors, low-cost volumetric capture systems are easier to deploy. Their wider adoption though depends on their usability and by extension on the practicality of spatially aligning multiple sensors. Most existing alignment approaches employ visual patterns, e.g. checkerboards, or markers and require high user involvement and technical knowledge. More user-friendly and easier-to-use approaches rely on markerless methods that exploit geometric patterns of a physical structure. However, current SoA approaches are bounded by restrictions in the placement and the number of sensors. In this work, we improve markerless data-driven correspondence estimation to achieve more robust and flexible multi-sensor spatial alignment. In particular, we incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. This is accomplished by a soft, differentiable procrustes analysis that regularizes the segmentation and achieves higher extrinsic calibration performance in expanded sensor placement configurations, while being unrestricted by the number of sensors of the volumetric capture system. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure. Code and pretrained models are available at https://vcl3d.github.io/StructureNet/.

Related papers

• Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes [56.52618054240197]
  We propose a novel, condition-aware multimodal fusion approach for robust semantic perception of driving scenes. Our method, CAFuser, uses an RGB camera input to classify environmental conditions and generate a Condition Token that guides the fusion of multiple sensor modalities. We set the new state of the art with CAFuser on the MUSES dataset with 59.7 PQ for multimodal panoptic segmentation and 78.2 mIoU for semantic segmentation, ranking first on the public benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T17:56:20Z)
• Geometry-aware Feature Matching for Large-Scale Structure from Motion [10.645087195983201]
  We introduce geometry cues in addition to color cues to fill gaps when there is less overlap in large-scale scenarios. Our approach ensures that the denser correspondences from detector-free methods are geometrically consistent and more accurate. It outperforms state-of-the-art feature matching methods on benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T21:41:35Z)
• Kalib: Markerless Hand-Eye Calibration with Keypoint Tracking [52.4190876409222]
  Hand-eye calibration involves estimating the transformation between the camera and the robot. Recent advancements in deep learning offer markerless techniques, but they present challenges. We propose Kalib, an automatic and universal markerless hand-eye calibration pipeline.
  arXiv  Detail & Related papers  (2024-08-20T06:03:40Z)
• SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields [10.958143040692141]
  In rapidly-evolving domains such as autonomous driving, the use of multiple sensors with different modalities is crucial to ensure operational precision and stability. To correctly exploit the provided information by each sensor in a single common frame, it is essential for these sensors to be accurately calibrated. We leverage the ability of Neural Radiance Fields to represent different modalities in a common representation.
  arXiv  Detail & Related papers  (2023-11-27T13:25:47Z)
• Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction [82.72686460985297]
  We tackle the problem of estimating a Manhattan frame. We derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers. We also design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization.
  arXiv  Detail & Related papers  (2023-08-21T13:03:25Z)
• Instance Segmentation with Cross-Modal Consistency [13.524441194366544]
  We introduce a novel approach to instance segmentation that jointly leverages measurements from multiple sensor modalities. Our technique applies contrastive learning to points in the scene both across sensor modalities and the temporal domain. We demonstrate that this formulation encourages the models to learn embeddings that are invariant to viewpoint variations.
  arXiv  Detail & Related papers  (2022-10-14T21:17:19Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Dense Label Encoding for Boundary Discontinuity Free Rotation Detection [69.75559390700887]
  This paper explores a relatively less-studied methodology based on classification. We propose new techniques to push its frontier in two aspects. Experiments and visual analysis on large-scale public datasets for aerial images show the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-11-19T05:42:02Z)
• Deep Phase Correlation for End-to-End Heterogeneous Sensor Measurements Matching [12.93459392278491]
  We present an end-to-end deep phase correlation network (DPCN) to match heterogeneous sensor measurements. The primary component is a differentiable correlation-based estimator that back-propagates the pose error to learnable feature extractors. With the interpretable modeling, the network is light-weighted and promising for better generalization.
  arXiv  Detail & Related papers  (2020-08-21T13:42:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.