MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception

• URL: http://arxiv.org/abs/2609.02717v1
• Date: Wed, 02 Sep 2026 15:25:28 GMT
• Title: MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception
• Abstract summary: MV-dVRK is the first ex-vivo surgical dataset to combine multiple exposure-synchronized stereo viewpoints with accurate surface geometry and camera poses.<n>We use MV-dVRK to systematically compare zero-shot monocular, stereo, multi-stereo, and multi-view 3D reconstruction methods as the number of viewpoints increases.
• Score: 15.926513958030148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale training and refined optimization techniques have greatly improved sparse multi-view 3D reconstruction. Despite their relevance to surgery, such methods have never before been rigorously evaluated on real endoscopic images. Current clinical telerobots deploy a single stereo camera inside the patient, making multi-viewpoint data extremely rare. This paper presents MV-dVRK, the first ex-vivo surgical dataset to combine multiple exposure-synchronized stereo viewpoints with accurate surface geometry and camera poses. The static subset of the benchmark provides dense SfM reference geometry, validated against an industrial 3D scanner, together with ground-truth camera poses and sparse-view test sets. We use MV-dVRK to systematically compare zero-shot monocular, stereo, multi-stereo, and multi-view 3D reconstruction methods as the number of viewpoints increases. With two endoscopes, multi-stereo reconstruction achieves the highest coverage. With a third viewpoint, optimization-based multi-view methods perform best, covering 67% of ground-truth surface points within a 1 mm tolerance and recovering highly accurate relative camera poses. By contrast, feed-forward foundation models cover only 43% of the ground-truth surface in the same setting. MV-dVRK also includes ten dynamic sequences spanning multiple surgical tasks, with increasing kinematic complexity and tissue deformation, providing a basis for future research in multi-viewpoint surgical perception. The project is available at: https://mv-dvrk.is.mpg.de.

Related papers

• Multiview Multi-Person Human Mesh Recovery Under Large Scenes with Occlusions [53.539496609429015]
  We introduce a large-scale synthetic benchmark for multiview multi-person HMR, termed MVMP-HMR.<n>Based on this benchmark, we propose a multiview multi-person whole-body human mesh recovery model, referred to as MVMP-HMR.<n>The model first fuses multiview features into a scene-level 3D feature volume, and then leverages pelvis joints predicted by a 3D pose estimation network to extract person-specific queries from the 3D feature volume.<n>These human queries are cross-attended with the 3D feature volume and integrated to decode each person's 3D mesh.
  arXiv  Detail & Related papers  (2026-07-27T11:47:26Z)
• Learning Surgical Robotic Manipulation with 3D Spatial Priors [73.00031539525202]
  We introduce the Spatial Surgical Transformer (SST), an end-to-end visuomotor policy that empowers surgical robots with 3D spatial awareness.<n>Based on Surgical3D, we finetune a powerful geometric transformer to extract robust 3D latent representations from stereo endoscopes images.<n>SST achieves state-of-the-art performance and strong spatial generalization on complex surgical tasks.
  arXiv  Detail & Related papers  (2026-03-04T07:19:56Z)
• EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups [76.30400481133215]
  Multiview photometric stereo (MVPS) and multiview shape from polarization (MVSfP) have not been quantitatively assessed together with MVS.<n>We propose EvalMVX, a real-world dataset containing $25$ objects, each captured with a polarized camera under $20$ varying views.<n>We evaluate $13$ MVX methods published in recent years, record the best-performing methods, and identify open problems under diverse geometric details.
  arXiv  Detail & Related papers  (2026-02-27T14:53:29Z)
• Post-mastoidectomy Surface Multi-View Synthesis from a Single Microscopy Image [4.777201894011511]
  We introduce a novel pipeline that is capable of generating synthetic multi-view videos from a single CI microscope image.<n>We use a patient's pre-operative CT scan to predict the post-mastoidectomy surface using a method designed for this purpose.
  arXiv  Detail & Related papers  (2024-08-31T16:45:24Z)
• DUSt3R: Geometric 3D Vision Made Easy [8.471330244002564]
  We introduce DUSt3R, a novel paradigm for Dense and Unconstrained Stereo 3D Reconstruction of arbitrary image collections.<n>We show that this formulation smoothly unifies the monocular and binocular reconstruction cases.<n>Our formulation directly provides a 3D model of the scene as well as depth information, but interestingly, we can seamlessly recover from it, pixel matches, relative and absolute camera.
  arXiv  Detail & Related papers  (2023-12-21T18:52:14Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• SERV-CT: A disparity dataset from CT for validation of endoscopic 3D reconstruction [8.448866668577946]
  We present a stereo-endoscopic reconstruction validation dataset based on CT (SERV-CT) The SERV-CT dataset provides an easy to use stereoscopic validation for surgical applications with smooth reference disparities and depths with coverage over the majority of the endoscopic images.
  arXiv  Detail & Related papers  (2020-12-22T01:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.