Deep learning with 4D spatio-temporal data representations for OCT-based force estimation

• URL: http://arxiv.org/abs/2005.10033v1
• Date: Wed, 20 May 2020 13:30:36 GMT
• Title: Deep learning with 4D spatio-temporal data representations for OCT-based force estimation
• Authors: Nils Gessert, Marcel Bengs, Matthias Schl\"uter, and Alexander Schlaefer
• Abstract summary: We extend the problem of deep learning-based force estimation to 4D volumetric-temporal data with streams of 3D OCT volumes. We show that using 4Dterm-temporal data outperforms all previously used data representations with a mean absolute error of 10.7mN.
• Score: 59.405210617831656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimating the forces acting between instruments and tissue is a challenging problem for robot-assisted minimally-invasive surgery. Recently, numerous vision-based methods have been proposed to replace electro-mechanical approaches. Moreover, optical coherence tomography (OCT) and deep learning have been used for estimating forces based on deformation observed in volumetric image data. The method demonstrated the advantage of deep learning with 3D volumetric data over 2D depth images for force estimation. In this work, we extend the problem of deep learning-based force estimation to 4D spatio-temporal data with streams of 3D OCT volumes. For this purpose, we design and evaluate several methods extending spatio-temporal deep learning to 4D which is largely unexplored so far. Furthermore, we provide an in-depth analysis of multi-dimensional image data representations for force estimation, comparing our 4D approach to previous, lower-dimensional methods. Also, we analyze the effect of temporal information and we study the prediction of short-term future force values, which could facilitate safety features. For our 4D force estimation architectures, we find that efficient decoupling of spatial and temporal processing is advantageous. We show that using 4D spatio-temporal data outperforms all previously used data representations with a mean absolute error of 10.7mN. We find that temporal information is valuable for force estimation and we demonstrate the feasibility of force prediction.

Related papers

• Iterative Occlusion-Aware Light Field Depth Estimation using 4D Geometrical Cues [0.0]
  This paper focuses on explicitly understanding and exploiting 4D geometric cues for light field depth estimation. A 4D model performs depth/disparity estimation by determining the orientations and analysing the intersections of key 2D planes in 4D space. Experimental results show that the proposed method outperforms both learning-based and non-learning-based state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-04T13:47:49Z)
• H4D: Human 4D Modeling by Learning Neural Compositional Representation [75.34798886466311]
  This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation. Experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks.
  arXiv  Detail & Related papers  (2022-03-02T17:10:49Z)
• Magnifying Subtle Facial Motions for Effective 4D Expression Recognition [56.806738404887824]
  The flow of 3D faces is first analyzed to capture the spatial deformations. The obtained temporal evolution of these deformations are fed into a magnification method. The latter, main contribution of this paper, allows revealing subtle (hidden) deformations which enhance the emotion classification performance.
  arXiv  Detail & Related papers  (2021-05-05T20:47:43Z)
• 3D Scattering Tomography by Deep Learning with Architecture Tailored to Cloud Fields [12.139158398361866]
  We present 3DeepCT, a deep neural network for computed tomography, which performs 3D reconstruction of scattering volumes from multi-view images. We show that 3DeepCT outperforms physics-based inverse scattering methods in term of accuracy as well as offering a significant orders of magnitude improvement in computational time.
  arXiv  Detail & Related papers  (2020-12-10T20:31:44Z)
• 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes [69.62333053044712]
  3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
  arXiv  Detail & Related papers  (2020-07-02T12:02:20Z)
• A Deep Learning Approach for Motion Forecasting Using 4D OCT Data [69.62333053044712]
  We propose 4D-temporal deep learning for end-to-end motion forecasting and estimation using a stream of OCT volumes. Our best performing 4D method achieves motion forecasting with an overall average correlation of 97.41%, while also improving motion estimation performance by a factor of 2.5 compared to a previous 3D approach.
  arXiv  Detail & Related papers  (2020-04-21T15:59:53Z)
• Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT Image Data [63.73263986460191]
  Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. Using 4D information for the model input improves performance while maintaining reasonable inference times.
  arXiv  Detail & Related papers  (2020-04-21T15:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.