Deep Learning based Super-Resolution for Medical Volume Visualization with Direct Volume Rendering

• URL: http://arxiv.org/abs/2210.08080v1
• Date: Fri, 14 Oct 2022 19:58:59 GMT
• Title: Deep Learning based Super-Resolution for Medical Volume Visualization with Direct Volume Rendering
• Authors: Sudarshan Devkota, Sumanta Pattanaik
• Abstract summary: Recent advances in deep learning-based image and video super-resolution techniques motivate us to investigate such networks for high-fidelity upscaling of frames rendered at a lower resolution to a higher resolution. We propose a technique where our proposed system uses color information along with other medical features gathered from our volume to learn efficient upscaling of a low-resolution rendering to a higher-resolution space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern-day display systems demand high-quality rendering. However, rendering at higher resolution requires a large number of data samples and is computationally expensive. Recent advances in deep learning-based image and video super-resolution techniques motivate us to investigate such networks for high-fidelity upscaling of frames rendered at a lower resolution to a higher resolution. While our work focuses on super-resolution of medical volume visualization performed with direct volume rendering, it is also applicable for volume visualization with other rendering techniques. We propose a learning-based technique where our proposed system uses color information along with other supplementary features gathered from our volume renderer to learn efficient upscaling of a low-resolution rendering to a higher-resolution space. Furthermore, to improve temporal stability, we also implement the temporal reprojection technique for accumulating history samples in volumetric rendering.

Related papers

• Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Network [8.739451985459638]
  Super-resolution algorithms transform one or more sets of low-resolution images captured from the same scene into high-resolution images. The extraction of image features and nonlinear mapping methods in the reconstruction process remain challenging for existing algorithms. The objective is to recover high-quality, high-resolution images from low-resolution images.
  arXiv  Detail & Related papers  (2024-07-18T06:50:39Z)
• Auxiliary Features-Guided Super Resolution for Monte Carlo Rendering [8.54858933529271]
  Super resolution to reduce the number of pixels to render and thus speed up Monte Carlo rendering algorithms. We exploit high-resolution auxiliary features to guide super resolution of low-resolution renderings. Our experiments show that our auxiliary features-guided super-resolution method outperforms both super-resolution methods and Monte Carlo denoising methods in producing high-quality renderings.
  arXiv  Detail & Related papers  (2023-10-20T02:45:13Z)
• ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models [126.35334860896373]
  We investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes. Existing works for higher-resolution generation, such as attention-based and joint-diffusion approaches, cannot well address these issues. We propose a simple yet effective re-dilation that can dynamically adjust the convolutional perception field during inference.
  arXiv  Detail & Related papers  (2023-10-11T17:52:39Z)
• Super-Resolving Face Image by Facial Parsing Information [52.1267613768555]
  Face super-resolution is a technology that transforms a low-resolution face image into the corresponding high-resolution one. We build a novel parsing map guided face super-resolution network which extracts the face prior from low-resolution face image. High-resolution features contain more precise spatial information while low-resolution features provide strong contextual information.
  arXiv  Detail & Related papers  (2023-04-06T08:19:03Z)
• FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks [33.489890950757975]
  FoVolNet is a method to significantly increase the performance of volume data visualization. We develop a cost-effective foveated rendering pipeline that sparsely samples a volume around a focal point and reconstructs the full-frame using a deep neural network.
  arXiv  Detail & Related papers  (2022-09-20T19:48:56Z)
• Efficient High-Resolution Deep Learning: A Survey [90.76576712433595]
  Cameras in modern devices such as smartphones, satellites and medical equipment are capable of capturing very high resolution images and videos. Such high-resolution data often need to be processed by deep learning models for cancer detection, automated road navigation, weather prediction, surveillance, optimizing agricultural processes and many other applications. Using high-resolution images and videos as direct inputs for deep learning models creates many challenges due to their high number of parameters, computation cost, inference latency and GPU memory consumption. Several works in the literature propose better alternatives in order to deal with the challenges of high-resolution data and improve accuracy and speed while complying with hardware limitations
  arXiv  Detail & Related papers  (2022-07-26T17:13:53Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images [57.253447453301796]
  We introduce Deep Direct Volume Rendering (DeepDVR), a generalization of Direct Volume Rendering (DVR) that allows for the integration of deep neural networks into the DVR algorithm. We conceptualize the rendering in a latent color space, thus enabling the use of deep architectures to learn implicit mappings for feature extraction and classification. Our generalization serves to derive novel volume rendering architectures that can be trained end-to-end directly from examples in image space.
  arXiv  Detail & Related papers  (2021-06-09T23:03:00Z)
• Zoom-to-Inpaint: Image Inpainting with High-Frequency Details [39.582275854002994]
  We propose applying super-resolution to coarsely reconstructed outputs, refining them at high resolution, and then downscaling the output to the original resolution. By introducing high-resolution images to the refinement network, our framework is able to reconstruct finer details that are usually smoothed out due to spectral bias. Our zoom-in, refine and zoom-out strategy, combined with high-resolution supervision and progressive learning, constitutes a framework-agnostic approach for enhancing high-frequency details.
  arXiv  Detail & Related papers  (2020-12-17T05:39:37Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.