Markerless 6D Pose Estimation and Position-Based Visual Servoing for Endoscopic Continuum Manipulators

• URL: http://arxiv.org/abs/2602.16365v1
• Date: Wed, 18 Feb 2026 11:08:32 GMT
• Title: Markerless 6D Pose Estimation and Position-Based Visual Servoing for Endoscopic Continuum Manipulators
• Abstract summary: This paper presents a unified framework for markerless stereo 6D pose estimation and position-based visual servoing of continuum manipulators.<n>A photo-realistic simulation pipeline enables large-scale automatic training with pixel-accurate annotations.<n>To the best of our knowledge, this work presents the first fully markerless pose-estimation-driven position-based visual servoing framework for continuum manipulators.
• Score: 3.324399280473623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continuum manipulators in flexible endoscopic surgical systems offer high dexterity for minimally invasive procedures; however, accurate pose estimation and closed-loop control remain challenging due to hysteresis, compliance, and limited distal sensing. Vision-based approaches reduce hardware complexity but are often constrained by limited geometric observability and high computational overhead, restricting real-time closed-loop applicability. This paper presents a unified framework for markerless stereo 6D pose estimation and position-based visual servoing of continuum manipulators. A photo-realistic simulation pipeline enables large-scale automatic training with pixel-accurate annotations. A stereo-aware multi-feature fusion network jointly exploits segmentation masks, keypoints, heatmaps, and bounding boxes to enhance geometric observability. To enforce geometric consistency without iterative optimization, a feed-forward rendering-based refinement module predicts residual pose corrections in a single pass. A self-supervised sim-to-real adaptation strategy further improves real-world performance using unlabeled data. Extensive real-world validation achieves a mean translation error of 0.83 mm and a mean rotation error of 2.76° across 1,000 samples. Markerless closed-loop visual servoing driven by the estimated pose attains accurate trajectory tracking with a mean translation error of 2.07 mm and a mean rotation error of 7.41°, corresponding to 85% and 59% reductions compared to open-loop control, together with high repeatability in repeated point-reaching tasks. To the best of our knowledge, this work presents the first fully markerless pose-estimation-driven position-based visual servoing framework for continuum manipulators, enabling precise closed-loop control without physical markers or embedded sensing.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes [55.15267980622371]
  RRTrack is an efficient, recoverable object 6D pose tracker.<n>It enables robust tracking through fast motion and target disappearance--reappearance.<n>Experiments show RRTrack improves equal-subset mean ADD-S AR by 66.3% and ADD-S AUC by 65.7% over FoundationPose.
  arXiv  Detail & Related papers  (2026-07-26T14:05:46Z)
• InvariantCloud: A Globally Invariant, Uniquely Indexed Point Cloud Framework for Robust 6-DoF Tactile Pose Tracking [7.9092948142021]
  We introduce InvariantCloud, a 6-DoF tactile object pose estimation framework.<n>We show that InvariantCloud achieves superior yaw tracking accuracy and re-localization repeatability compared to existing benchmarks.
  arXiv  Detail & Related papers  (2026-05-24T18:46:24Z)
• CT-VIR: Continuous-Time Visual-Inertial-Ranging Fusion for Indoor Localization with Sparse Anchors [8.090575303185078]
  We propose a continuous-time state estimation method for visual-inertial-ranging (VIR) fusion localization.<n>Introductor, visual, and ranging constraints are formulated as factors in a sliding-window graph.<n> Evaluations on public datasets and real-world experiments demonstrate the effectiveness and practical potential of the proposed approach.
  arXiv  Detail & Related papers  (2026-04-16T02:23:57Z)
• Stereo-Inertial Poser: Towards Metric-Accurate Shape-Aware Motion Capture Using Sparse IMUs and a Single Stereo Camera [54.967647497048205]
  We present Stereo-Inertial Poser, a real-time motion capture system that estimates metric-accurate and shape-aware 3D human motion.<n>We replace the monocular RGB with stereo vision, enabling direct 3D keypoint extraction and body shape parameter estimation.<n>Our method produces drift-free global translation under a long recording time and reduces foot-skating effects.
  arXiv  Detail & Related papers  (2026-03-02T17:46:38Z)
• MLRecon: Robust Markerless Freehand 3D Ultrasound Reconstruction via Coarse-to-Fine Pose Estimation [8.489758968497188]
  We present MLRecon, a robust markerless 3D US reconstruction framework delivering drift-resilient 6D probe pose tracking.<n>Our pipeline enables continuous markerless tracking of the probe, augmented by a vision-guided divergence detector.<n>Experiments demonstrate that MLRecon significantly outperforms competing sensorless and sensor-aided methods.
  arXiv  Detail & Related papers  (2026-03-01T08:39:21Z)
• Geometry OR Tracker: Universal Geometric Operating Room Tracking [61.399734016038614]
  In operating rooms (OR), world-scale multi-view 3D tracking supports downstream applications such as surgeon behavior recognition.<n>Camera calibration and RGB-D registration are always unreliable, leading to cross-view geometric inconsistency.<n>We introduce Geometry OR Tracker, a two-stage pipeline that rectifies imprecise calibration into a scaleconsistent and geometrically consistent camera setup.
  arXiv  Detail & Related papers  (2026-02-28T09:21:21Z)
• A2VISR: An Active and Adaptive Ground-Aerial Localization System Using Visual Inertial and Single-Range Fusion [17.59898575353575]
  It's a practical approach using the ground-aerial collaborative system to enhance the localization robustness of flying robots in cluttered environments.<n>We improve the ground-aerial localization framework in a more comprehensive manner, which integrates active vision, single-ranging, inertialometry, and optical flow.<n>The proposed approach achieves robust online localization, with an average root mean square error of approximately 0.09 m, while maintaining resilience to capture loss and sensor failures.
  arXiv  Detail & Related papers  (2025-12-18T10:07:06Z)
• Visual Odometry with Transformers [68.453547770334]
  We introduce Visual odometry Transformer (VoT), which processes sequences of monocular frames by extracting features.<n>Unlike prior methods, VoT directly predicts camera motion without estimating dense geometry and relies solely on camera poses for supervision.<n>VoT scales effectively with larger datasets, benefits substantially from stronger pre-trained backbones, generalizes across diverse camera motions and calibration settings, and outperforms traditional methods while running more than 3 times faster.
  arXiv  Detail & Related papers  (2025-10-02T17:00:14Z)
• Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices [4.261261166281339]
  We present a unified framework explicitly designed for efficient execution on edge devices.<n>Key to our approach is a shared, lighting-invariant color-pair feature representation.<n>For initial estimation, this feature facilitates robust registration between the live RGB-D view and the object's 3D mesh.<n>For tracking, the same feature logic validates temporal correspondences, enabling a lightweight model to reliably regress the object's motion.
  arXiv  Detail & Related papers  (2025-09-28T05:07:49Z)
• Simulating Sinogram-Domain Motion and Correcting Image-Domain Artifacts Using Deep Learning in HR-pQCT Bone Imaging [0.0]
  We optimize a conventional sinogram-based method to simulate motion artifacts in HR-pQCT images.<n>We propose an Edge-enhanced Self-attention Wasserstein Generative Adversarial Network with Gradient Penalty (ESWGAN-GP) to address motion artifacts in both simulated (source) and real-world (target) datasets.
  arXiv  Detail & Related papers  (2025-09-13T19:43:05Z)
• UniCalib: Targetless LiDAR-Camera Calibration via Probabilistic Flow on Unified Depth Representations [30.56092814783138]
  DF-Calib is a LiDAR-camera calibration method that reformulates calibration as an intra-modality depth flow estimation problem.<n> DF-Calib estimates a dense depth map from the camera image and completes the sparse LiDAR projected depth map.<n>We introduce a reliability map to prioritize valid pixels and propose a perceptually weighted sparse flow loss to enhance depth flow estimation.
  arXiv  Detail & Related papers  (2025-04-02T07:09:44Z)
• CalibRefine: Deep Learning-Based Online Automatic Targetless LiDAR-Camera Calibration with Iterative and Attention-Driven Post-Refinement [7.736775961390864]
  CalibRefine is a fully automatic, targetless, and online calibration framework.<n>It directly processes raw LiDAR point clouds and camera images.<n>Our results show that robust object-level feature matching, combined with iterative refinement and self-supervised attention-based refinement, enables reliable sensor alignment.
  arXiv  Detail & Related papers  (2025-02-24T20:53:42Z)
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [56.66677293607114]
  We propose Code-as-Monitor (CaM) for both open-set reactive and proactive failure detection.<n>To enhance the accuracy and efficiency of monitoring, we introduce constraint elements that abstract constraint-related entities.<n>Experiments show that CaM achieves a 28.7% higher success rate and reduces execution time by 31.8% under severe disturbances.
  arXiv  Detail & Related papers  (2024-12-05T18:58:27Z)
• Improving Gaussian Splatting with Localized Points Management [52.009874685460694]
  Localized Point Management (LPM) is capable of identifying those error-contributing zones in greatest need for both point addition and geometry calibration.<n>LPM applies point densification in the identified zones and then reset the opacity of the points in front of these regions, creating a new opportunity to correct poorly conditioned points.<n> Notably, LPM improves both static 3DGS and dynamic SpaceTimeGS to achieve state-of-the-art rendering quality while retaining real-time speeds.
  arXiv  Detail & Related papers  (2024-06-06T16:55:07Z)
• DeepRM: Deep Recurrent Matching for 6D Pose Refinement [77.34726150561087]
  DeepRM is a novel recurrent network architecture for 6D pose refinement. The architecture incorporates LSTM units to propagate information through each refinement step. DeepRM achieves state-of-the-art performance on two widely accepted challenging datasets.
  arXiv  Detail & Related papers  (2022-05-28T16:18:08Z)
• Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping [58.50342759993186]
  We propose a visuo-tactile model for realtime estimation of the liquid inside a deformable container. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor. The robotic system is well controlled and adjusted based on the estimation model in real time.
  arXiv  Detail & Related papers  (2022-02-23T13:38:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.