Mixed Reality using Illumination-aware Gradient Mixing in Surgical Telepresence: Enhanced Multi-layer Visualization

• URL: http://arxiv.org/abs/2110.09318v1
• Date: Sat, 21 Aug 2021 11:59:24 GMT
• Title: Mixed Reality using Illumination-aware Gradient Mixing in Surgical Telepresence: Enhanced Multi-layer Visualization
• Abstract summary: The aim of this work is to propose a solution to improve the visualization in the final merged video by producing globally consistent videos. The proposed system uses an enhanced multi-layer visualization with illumination-aware gradient mixing.
• Score: 27.681891555949672
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Background and aim: Surgical telepresence using augmented perception has been applied, but mixed reality is still being researched and is only theoretical. The aim of this work is to propose a solution to improve the visualization in the final merged video by producing globally consistent videos when the intensity of illumination in the input source and target video varies. Methodology: The proposed system uses an enhanced multi-layer visualization with illumination-aware gradient mixing using Illumination Aware Video Composition algorithm. Particle Swarm Optimization Algorithm is used to find the best sample pair from foreground and background region and image pixel correlation to estimate the alpha matte. Particle Swarm Optimization algorithm helps to get the original colour and depth of the unknown pixel in the unknown region. Result: Our results showed improved accuracy caused by reducing the Mean squared Error for selecting the best sample pair for unknown region in 10 each sample for bowel, jaw and breast. The amount of this reduction is 16.48% from the state of art system. As a result, the visibility accuracy is improved from 89.4 to 97.7% which helped to clear the hand vision even in the difference of light. Conclusion: Illumination effect and alpha pixel correlation improves the visualization accuracy and produces a globally consistent composition results and maintains the temporal coherency when compositing two videos with high and inverse illumination effect. In addition, this paper provides a solution for selecting the best sampling pair for the unknown region to obtain the original colour and depth.

Related papers

• Depth-Supervised Fusion Network for Seamless-Free Image Stitching [17.758594808190264]
  Image stitching synthesizes images captured from multiple perspectives into a single image with a broader field of view.<n>The significant variations in object depth often lead to large parallax, resulting in ghosting and misalignment in the stitched results.<n>To address this, we propose a depth-consistency-constrained seamless-free image stitching method.
  arXiv  Detail & Related papers  (2025-10-24T12:36:08Z)
• Materialist: Physically Based Editing Using Single-Image Inverse Rendering [50.39048790589746]
  We present a method combining a learning-based approach with progressive differentiable rendering. Our method achieves more realistic light material interactions, accurate shadows, and global illumination. We also propose a method for material transparency editing that operates effectively without requiring full scene geometry.
  arXiv  Detail & Related papers  (2025-01-07T11:52:01Z)
• Zero-Shot Enhancement of Low-Light Image Based on Retinex Decomposition [4.175396687130961]
  We propose a new learning-based Retinex decomposition of zero-shot low-light enhancement method, called ZERRINNet. Our method is a zero-reference enhancement method that is not affected by the training data of paired and unpaired datasets.
  arXiv  Detail & Related papers  (2023-11-06T09:57:48Z)
• Guided Linear Upsampling [8.819059777836628]
  Guided upsampling is an effective approach for accelerating high-resolution image processing. Our method can better preserve detail effects while suppressing artifacts such as bleeding and blurring. We demonstrate the advantages of our method for both interactive image editing and real-time high-resolution video processing.
  arXiv  Detail & Related papers  (2023-07-13T08:04:24Z)
• LUT-GCE: Lookup Table Global Curve Estimation for Fast Low-light Image Enhancement [62.17015413594777]
  We present an effective and efficient approach for low-light image enhancement, named LUT-GCE. We estimate a global curve for the entire image that allows corrections for both under- and over-exposure. Our approach outperforms the state of the art in terms of inference speed, especially on high-definition images (e.g., 1080p and 4k)
  arXiv  Detail & Related papers  (2023-06-12T12:53:06Z)
• Exposure Fusion for Hand-held Camera Inputs with Optical Flow and PatchMatch [53.149395644547226]
  We propose a hybrid synthesis method for multi-exposure image fusion taken by hand-held cameras. Our method can deal with such motions and maintain the exposure information of each input effectively. Experiment results demonstrate the effectiveness and robustness of our method.
  arXiv  Detail & Related papers  (2023-04-10T09:06:37Z)
• NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination [48.42173911185454]
  Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images. We propose an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images.
  arXiv  Detail & Related papers  (2023-03-29T12:05:19Z)
• NLHD: A Pixel-Level Non-Local Retinex Model for Low-Light Image Enhancement [7.489618301273569]
  We propose a new pixel-level non-local Haar transform based illumination and reflectance decomposition method (NLHD) The complete similarity of pixels in a matched similar pixel group and the simple separable Haar transform help to obtain more appropriate image decomposition. Experiments on benchmark datasets show that the proposed method can achieve better low-light image enhancement results on subjective and objective evaluations.
  arXiv  Detail & Related papers  (2021-06-13T11:48:14Z)
• Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
  Differentiable geometric computation has received increasing interest for image-based inverse problems. We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation. As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
  arXiv  Detail & Related papers  (2021-04-01T09:29:05Z)
• Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement [52.49231695707198]
  We investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps. Inspired by the color image formulation, we first estimate the degradation from low-light inputs to simulate the distortion of environment illumination color, and then refine the content to recover the loss of diffuse illumination color. Our proposed method has surpassed the SOTA by 0.95dB in PSNR on LOL1000 dataset and 3.18% in mAP on ExDark dataset.
  arXiv  Detail & Related papers  (2021-03-19T04:00:27Z)
• A Novel Solution of Using Mixed Reality in Bowel and Oral and Maxillofacial Surgical Telepresence: 3D Mean Value Cloning algorithm [29.231808429620646]
  The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and the virtual hand of the remote expertise surgeon. The proposed system enhanced the mean value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video.
  arXiv  Detail & Related papers  (2021-03-17T10:01:06Z)
• Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising [104.59305271099967]
  We present a pixel aggregation network and learn the pixel sampling and averaging strategies for image denoising. We develop a pixel aggregation network for video denoising to sample pixels across the spatial-temporal space. Our method is able to solve the misalignment issues caused by large motion in dynamic scenes.
  arXiv  Detail & Related papers  (2021-01-26T13:00:46Z)
• Low-light Image Enhancement Using the Cell Vibration Model [12.400040803969501]
  Low light very likely leads to the degradation of an image's quality and even causes visual task failures. We propose a new single low-light image lightness enhancement method. Experimental results show that the proposed algorithm is superior to nine state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-03T13:39:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.