Correct-by-Construction Vision-based Pose Estimation using Geometric Generative Models

• URL: http://arxiv.org/abs/2601.17556v1
• Date: Sat, 24 Jan 2026 18:57:38 GMT
• Title: Correct-by-Construction Vision-based Pose Estimation using Geometric Generative Models
• Abstract summary: We present a framework for designing certifiable neural networks (NNs) for perception-based pose estimation.<n>We first demonstrate this framework in uncluttered environments, where the target object is the only object present in the camera's field of view.<n>We extend this using ideas from NN reachability analysis to design certified object NN that can detect the presence of the target object in cluttered environments.
• Score: 4.282159812965446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of vision-based pose estimation for autonomous systems. While deep neural networks have been successfully used for vision-based tasks, they inherently lack provable guarantees on the correctness of their output, which is crucial for safety-critical applications. We present a framework for designing certifiable neural networks (NNs) for perception-based pose estimation that integrates physics-driven modeling with learning-based estimation. The proposed framework begins by leveraging the known geometry of planar objects commonly found in the environment, such as traffic signs and runway markings, referred to as target objects. At its core, it introduces a geometric generative model (GGM), a neural-network-like model whose parameters are derived from the image formation process of a target object observed by a camera. Once designed, the GGM can be used to train NN-based pose estimators with certified guarantees in terms of their estimation errors. We first demonstrate this framework in uncluttered environments, where the target object is the only object present in the camera's field of view. We extend this using ideas from NN reachability analysis to design certified object NN that can detect the presence of the target object in cluttered environments. Subsequently, the framework consolidates the certified object detector with the certified pose estimator to design a multi-stage perception pipeline that generalizes the proposed approach to cluttered environments, while maintaining its certified guarantees. We evaluate the proposed framework using both synthetic and real images of various planar objects commonly encountered by autonomous vehicles. Using images captured by an event-based camera, we show that the trained encoder can effectively estimate the pose of a traffic sign in accordance with the certified bound provided by the framework.

Related papers

• Object Pose Transformer: Unifying Unseen Object Pose Estimation [54.20344997573707]
  Learning model-free object pose estimation for unseen instances remains a fundamental challenge in 3D vision.<n>ours jointly predicts depth, point maps, camera parameters, and normalized object coordinates from RGB inputs.<n>ours is camera-agnostic, learning camera intrinsics on-the-fly and supporting optional depth input for metric-scale recovery.
  arXiv  Detail & Related papers  (2026-03-24T16:04:16Z)
• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding [2.1987601456703474]
  We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation.<n>Our system first employs the CNOS detector to localize target objects.<n>For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose.
  arXiv  Detail & Related papers  (2025-11-16T14:19:52Z)
• Verification of Visual Controllers via Compositional Geometric Transformations [49.81690518952909]
  We introduce a novel verification framework for perception-based controllers that can generate outer-approximations of reachable sets.<n>We provide theoretical guarantees on the soundness of our method and demonstrate its effectiveness across benchmark control environments.
  arXiv  Detail & Related papers  (2025-07-06T20:22:58Z)
• BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimation [81.24160191975781]
  This paper presents a general RGB-based approach for object pose estimation, specifically designed to address challenges in sparse-view settings.<n>To overcome these limitations, we introduce corner points of the object bounding box as an intermediate representation of the object pose.<n>The 3D object corners can be reliably recovered from sparse input views, while the 2D corner points in the target view are estimated through a novel reference-based point datasets.
  arXiv  Detail & Related papers  (2025-04-10T17:58:35Z)
• Detecting Out-of-Distribution Objects through Class-Conditioned Inpainting [42.67743584239442]
  Real-world deployment often introduces novel and unexpected objects, referred to as out-of-distribution (OOD) objects.<n>Modern object detectors are typically overconfident, making it unreliable to use their predictions alone for OOD detection.<n>We utilize an off-the-shelf text-to-image generative model, such as Stable Diffusion, which is trained with objective functions distinct from those of discriminative object detectors.
  arXiv  Detail & Related papers  (2024-02-05T18:50:27Z)
• Context-Aware Indoor Point Cloud Object Generation through User Instructions [6.398660996031915]
  We present a novel end-to-end multi-modal deep neural network capable of generating point cloud objects seamlessly integrated with their surroundings. Our model revolutionizes scene modification by enabling the creation of new environments with previously unseen object layouts.
  arXiv  Detail & Related papers  (2023-11-26T06:40:16Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Towards Self-Supervised Category-Level Object Pose and Size Estimation [121.28537953301951]
  This work presents a self-supervised framework for category-level object pose and size estimation from a single depth image. We leverage the geometric consistency residing in point clouds of the same shape for self-supervision.
  arXiv  Detail & Related papers  (2022-03-06T06:02:30Z)
• Vision Checklist: Towards Testable Error Analysis of Image Models to Help System Designers Interrogate Model Capabilities [26.177391265710362]
  Vision Checklist is a framework aimed at interrogating the capabilities of a model in order to produce a report that can be used by a system designer for robustness evaluations. Our framework is evaluated on multiple datasets like Tinyimagenet, CIFAR10, CIFAR100 and Camelyon17 and for models like ViT and Resnet.
  arXiv  Detail & Related papers  (2022-01-27T17:20:16Z)
• Object-Driven Active Mapping for More Accurate Object Pose Estimation and Robotic Grasping [5.385583891213281]
  The framework is built on an object SLAM system integrated with a simultaneous multi-object pose estimation process. By combining the mapping module and the exploration strategy, an accurate object map that is compatible with robotic grasping can be generated.
  arXiv  Detail & Related papers  (2020-12-03T09:36:55Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.