A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training

• URL: http://arxiv.org/abs/2302.06019v2
• Date: Thu, 11 May 2023 18:46:39 GMT
• Title: A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training
• Authors: Jingnan Shi and Rajat Talak and Dominic Maggio and Luca Carlone
• Abstract summary: Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios. Our first contribution is to develop a robust corrector module that corrects pose estimates using depth information. Our second contribution is an ensemble self-training approach that simultaneously trains multiple pose estimators in a self-supervised manner.
• Score: 26.47895284071508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios. Modern learning-based approaches require large labeled datasets and tend to perform poorly outside the training domain. Our first contribution is to develop a robust corrector module that corrects pose estimates using depth information, thus enabling existing methods to better generalize to new test domains; the corrector operates on semantic keypoints (but is also applicable to other pose estimators) and is fully differentiable. Our second contribution is an ensemble self-training approach that simultaneously trains multiple pose estimators in a self-supervised manner. Our ensemble self-training architecture uses the robust corrector to refine the output of each pose estimator; then, it evaluates the quality of the outputs using observable correctness certificates; finally, it uses the observably correct outputs for further training, without requiring external supervision. As an additional contribution, we propose small improvements to a regression-based keypoint detection architecture, to enhance its robustness to outliers; these improvements include a robust pooling scheme and a robust centroid computation. Experiments on the YCBV and TLESS datasets show the proposed ensemble self-training outperforms fully supervised baselines while not requiring 3D annotations on real data.

Related papers

• VQA Training Sets are Self-play Environments for Generating Few-shot Pools [2.556825820539693]
  We propose a technique in which existing training sets can be directly used for constructing computational environments with task metrics as rewards. The proposed method starts with zero-shot prompts and iteratively refines them by selecting few-shot examples that maximize the task metric on the training set. Our experiments showcase how Gemini learns how to use itself, or another smaller and specialized model such as ScreenAI, to iteratively improve performance on training sets.
  arXiv  Detail & Related papers  (2024-05-30T07:38:58Z)
• Self-Supervised Dual Contouring [30.9409064656302]
  We propose a self-supervised training scheme for the Neural Dual Contouring meshing framework. We use two novel self-supervised loss functions that encourage consistency between distances to the generated mesh. We demonstrate that our self-supervised losses improve meshing performance in the single-view reconstruction task.
  arXiv  Detail & Related papers  (2024-05-28T12:44:28Z)
• Efficient Transferability Assessment for Selection of Pre-trained Detectors [63.21514888618542]
  This paper studies the efficient transferability assessment of pre-trained object detectors. We build up a detector transferability benchmark which contains a large and diverse zoo of pre-trained detectors. Experimental results demonstrate that our method outperforms other state-of-the-art approaches in assessing transferability.
  arXiv  Detail & Related papers  (2024-03-14T14:23:23Z)
• Consensus-Adaptive RANSAC [104.87576373187426]
  We propose a new RANSAC framework that learns to explore the parameter space by considering the residuals seen so far via a novel attention layer. The attention mechanism operates on a batch of point-to-model residuals, and updates a per-point estimation state to take into account the consensus found through a lightweight one-step transformer.
  arXiv  Detail & Related papers  (2023-07-26T08:25:46Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• Cost-Sensitive Self-Training for Optimizing Non-Decomposable Metrics [9.741019160068388]
  We introduce the Cost-Sensitive Self-Training (CSST) framework which generalizes the self-training-based methods for optimizing non-decomposable metrics. Our results demonstrate that CSST achieves an improvement over the state-of-the-art in majority of the cases across datasets and objectives.
  arXiv  Detail & Related papers  (2023-04-28T10:31:12Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training [23.802602957611676]
  We consider a certifiable object pose estimation problem, where -- given a partial point cloud of an object -- the goal is to provide a certificate of correctness for the resulting estimate. We propose C-3PO, a semantic-keypoint-based pose estimation model, augmented with the two certificates.
  arXiv  Detail & Related papers  (2022-06-22T17:06:39Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation [87.54604263202941]
  We propose a tiny deep neural network of which partial layers are iteratively exploited for refining its previous estimations. We employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model. Our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.
  arXiv  Detail & Related papers  (2021-11-11T23:31:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.