A Novel Solution of Using Mixed Reality in Bowel and Oral and Maxillofacial Surgical Telepresence: 3D Mean Value Cloning algorithm

• URL: http://arxiv.org/abs/2104.06316v1
• Date: Wed, 17 Mar 2021 10:01:06 GMT
• Title: A Novel Solution of Using Mixed Reality in Bowel and Oral and Maxillofacial Surgical Telepresence: 3D Mean Value Cloning algorithm
• Abstract summary: The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and the virtual hand of the remote expertise surgeon. The proposed system enhanced the mean value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video.
• Score: 29.231808429620646
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Background and aim: Most of the Mixed Reality models used in the surgical telepresence are suffering from discrepancies in the boundary area and spatial-temporal inconsistency due to the illumination variation in the video frames. The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and the virtual hand of the remote expertise surgeon. The purpose of the proposed solution is to decrease the processing time and enhance the accuracy of merged video by decreasing the overlay and visualization error and removing occlusion and artefacts. Methodology: The proposed system enhanced the mean value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video. The enhanced algorithm includes the 3D mean value coordinates and improvised mean value interpolant in the image cloning process, which helps to reduce the sawtooth, smudging and discolouration artefacts around the blending region. Results: As compared to the state of the art solution, the accuracy in terms of overlay error of the proposed solution is improved from 1.01mm to 0.80mm whereas the accuracy in terms of visualization error is improved from 98.8% to 99.4%. The processing time is reduced to 0.173 seconds from 0.211 seconds. Conclusion: Our solution helps make the object of interest consistent with the light intensity of the target image by adding the space distance that helps maintain the spatial consistency in the final merged video.

Related papers

• HiReFF: High-Resolution Feedforward Human Reconstruction from Uncalibrated Sparse-View Video [58.85548854304172]
  HiReFF is a feed-forward method for 2K-resolution 360 human video reconstruction from uncalibrated sparse-view videos.<n>Our framework decomposes the problem into two key tasks: 3D Gaussian reconstruction from sparse-view videos and computationally efficient high-resolution synthesis.<n> Experiments demonstrate that HiReFF significantly outperforms existing methods in high-resolution streaming video reconstruction.
  arXiv  Detail & Related papers  (2026-06-28T11:01:36Z)
• DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion [58.36400052566673]
  Infrared and visible image fusion integrates information from distinct spectral bands to enhance image quality. Existing approaches treat image fusion and subsequent high-level tasks as separate processes. We propose a Discriminative Cross- Dimension Evolutionary Learning Framework, termed DCEvo, which simultaneously enhances visual quality and perception accuracy.
  arXiv  Detail & Related papers  (2025-03-22T07:01:58Z)
• Temporal-Consistent Video Restoration with Pre-trained Diffusion Models [51.47188802535954]
  Video restoration (VR) aims to recover high-quality videos from degraded ones. Recent zero-shot VR methods using pre-trained diffusion models (DMs) suffer from approximation errors during reverse diffusion and insufficient temporal consistency. We present a novel a Posterior Maximum (MAP) framework that directly parameterizes video frames in the seed space of DMs, eliminating approximation errors.
  arXiv  Detail & Related papers  (2025-03-19T03:41:56Z)
• Spatiotemporal Multi-Camera Calibration using Freely Moving People [32.288669810272864]
  We propose a novel method for multi-camera calibration using freely moving people in multiview videos. We use 3D human poses obtained from an off-the-temporal monotemporal shelf to transform them into 3D points on a unit sphere. We employ a probabilistic approach that can jointly solve both problems of aligningtemporal data and establishing correspondences.
  arXiv  Detail & Related papers  (2025-02-18T05:15:52Z)
• VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [63.21396416244634]
  VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis. It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
  arXiv  Detail & Related papers  (2025-01-03T18:52:36Z)
• Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancement [48.76608212565327]
  This paper makes endeavors in the direction of learning for low-light video enhancement without using paired ground truth. Compared to low-light image enhancement, enhancing low-light videos is more difficult due to the intertwined effects of noise, exposure, and contrast in the spatial domain, jointly with the need for temporal coherence. We propose the Unrolled Decomposed Unpaired Network (UDU-Net) for enhancing low-light videos by unrolling the optimization functions into a deep network to decompose the signal into spatial and temporal-related factors, which are updated iteratively.
  arXiv  Detail & Related papers  (2024-08-22T11:45:11Z)
• Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution [151.1255837803585]
  We propose a novel approach, pursuing Spatial Adaptation and Temporal Coherence (SATeCo) for video super-resolution. SATeCo pivots on learning spatial-temporal guidance from low-resolution videos to calibrate both latent-space high-resolution video denoising and pixel-space video reconstruction. Experiments conducted on the REDS4 and Vid4 datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-03-25T17:59:26Z)
• Online estimation of the hand-eye transformation from surgical scenes [11.797350284719803]
  We present a neural network-based solution that estimates the transformation from a sequence of images and kinematic data. The proposed algorithm shows that the calibration procedure can be simplified by using deep learning techniques.
  arXiv  Detail & Related papers  (2023-06-04T04:55:02Z)
• Mixed Reality Depth Contour Occlusion Using Binocular Similarity Matching and Three-dimensional Contour Optimisation [3.9692358105634384]
  Mixed reality applications often require virtual objects that are partly occluded by real objects. Previous research and commercial products have limitations in terms of performance and efficiency.
  arXiv  Detail & Related papers  (2022-03-04T13:16:40Z)
• Differentiable Rendering with Perturbed Optimizers [85.66675707599782]
  Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Our work highlights the link between some well-known differentiable formulations and randomly smoothed renderings. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction.
  arXiv  Detail & Related papers  (2021-10-18T08:56:23Z)
• Mixed Reality using Illumination-aware Gradient Mixing in Surgical Telepresence: Enhanced Multi-layer Visualization [27.681891555949672]
  The aim of this work is to propose a solution to improve the visualization in the final merged video by producing globally consistent videos. The proposed system uses an enhanced multi-layer visualization with illumination-aware gradient mixing.
  arXiv  Detail & Related papers  (2021-08-21T11:59:24Z)
• Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV Images [53.11097060367591]
  We present a novel strategy for estimating an essential matrix from 360-FoV images in spherical projection. We show that our normalization can increase the camera pose accuracy by about 20% without significantly overhead the time.
  arXiv  Detail & Related papers  (2021-04-22T07:23:11Z)
• A Novel Visualization System of Using Augmented Reality in Knee Replacement Surgery: Enhanced Bidirectional Maximum Correntropy Algorithm [28.55685915431049]
  This study has improved video precision to 0.57 mm0.61 mm alignment error. With the use of bidirectional points, for example, forwards and backwards directional cloud point, the iteration on image registration was decreased.
  arXiv  Detail & Related papers  (2021-03-13T19:18:16Z)
• Enhanced Quadratic Video Interpolation [56.54662568085176]
  We propose an enhanced quadratic video (EQVI) model to handle more complicated scenes and motion patterns. To further boost the performance, we devise a novel multi-scale fusion network (MS-Fusion) which can be regarded as a learnable augmentation process. The proposed EQVI model won the first place in the AIM 2020 Video Temporal Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2020-09-10T02:31:50Z)
• Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild [49.672487902268706]
  We present a framework that jointly estimates camera temporal alignment and 3D point triangulation. We reconstruct 3D motion trajectories of human bodies in events captured by multiple unsynchronized and unsynchronized video cameras.
  arXiv  Detail & Related papers  (2020-07-24T23:50:46Z)
• Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video [10.181846237133167]
  We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time. The basic idea is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high resolution texture with an accuracy error of less than 2 mm.
  arXiv  Detail & Related papers  (2020-07-16T19:14:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.