4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes

• URL: http://arxiv.org/abs/2007.01044v1
• Date: Thu, 2 Jul 2020 12:02:20 GMT
• Title: 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes
• Authors: Marcel Bengs, Nils Gessert, Alexander Schlaefer
• Abstract summary: 3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
• Score: 69.62333053044712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tracking and localizing objects is a central problem in computer-assisted surgery. Optical coherence tomography (OCT) can be employed as an optical tracking system, due to its high spatial and temporal resolution. Recently, 3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single volumetric OCT images. While this approach relied on spatial information only, OCT allows for a temporal stream of OCT image volumes capturing the motion of an object at high volumes rates. In this work, we systematically extend 3D CNNs to 4D spatio-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking. Across various architectures, our results demonstrate that using a stream of OCT volumes and employing 4D spatio-temporal convolutions leads to a 30% lower mean absolute error compared to single volume processing with 3D CNNs.

Related papers

• μ-Net: A Deep Learning-Based Architecture for μ-CT Segmentation [2.012378666405002]
  X-ray computed microtomography (mu-CT) is a non-destructive technique that can generate high-resolution 3D images of the internal anatomy of medical and biological samples. extracting relevant information from 3D images requires semantic segmentation of the regions of interest. We propose a novel framework that uses a convolutional neural network (CNN) to automatically segment the full morphology of the heart of Carassius auratus.
  arXiv  Detail & Related papers  (2024-06-24T15:29:08Z)
• Distributed Stochastic Optimization of a Neural Representation Network for Time-Space Tomography Reconstruction [4.689071714940848]
  4D time-space reconstruction of dynamic events or deforming objects using Xray computed tomography (CT) is an extremely ill-posed inverse problem. Existing approaches assume that the object remains static for the duration of several tens or hundreds of X-ray projection measurement images. We propose to perform a 4D time-space reconstruction using a distributed implicit neural representation network that is trained using a novel distributed training algorithm.
  arXiv  Detail & Related papers  (2024-04-29T19:41:51Z)
• Physics-Informed Learning for Time-Resolved Angiographic Contrast Agent Concentration Reconstruction [3.3359894496511053]
  We present a neural network-based model that is trained on a dataset of image-based blood flow simulations. The model predicts the spatially averaged contrast agent concentration for each centerline point of the vasculature over time. Our approach demonstrates the potential of the integration of machine learning and blood flow simulations in time-resolved angiographic flow reconstruction.
  arXiv  Detail & Related papers  (2024-03-04T12:37:52Z)
• Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction [57.74609918453932]
  Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses. Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images. We argue that preserving the continuity of extracted vessels requires to take into account the global geometry. We propose a hybrid representation learning approach to address this challenge.
  arXiv  Detail & Related papers  (2020-12-14T05:22:49Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• Deep learning with 4D spatio-temporal data representations for OCT-based force estimation [59.405210617831656]
  We extend the problem of deep learning-based force estimation to 4D volumetric-temporal data with streams of 3D OCT volumes. We show that using 4Dterm-temporal data outperforms all previously used data representations with a mean absolute error of 10.7mN.
  arXiv  Detail & Related papers  (2020-05-20T13:30:36Z)
• A Deep Learning Approach for Motion Forecasting Using 4D OCT Data [69.62333053044712]
  We propose 4D-temporal deep learning for end-to-end motion forecasting and estimation using a stream of OCT volumes. Our best performing 4D method achieves motion forecasting with an overall average correlation of 97.41%, while also improving motion estimation performance by a factor of 2.5 compared to a previous 3D approach.
  arXiv  Detail & Related papers  (2020-04-21T15:59:53Z)
• Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT Image Data [63.73263986460191]
  Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. Using 4D information for the model input improves performance while maintaining reasonable inference times.
  arXiv  Detail & Related papers  (2020-04-21T15:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.