H-Net: A Multitask Architecture for Simultaneous 3D Force Estimation and Stereo Semantic Segmentation in Intracardiac Catheters

• URL: http://arxiv.org/abs/2501.00514v1
• Date: Tue, 31 Dec 2024 15:55:13 GMT
• Title: H-Net: A Multitask Architecture for Simultaneous 3D Force Estimation and Stereo Semantic Segmentation in Intracardiac Catheters
• Authors: Pedram Fekri, Mehrdad Zadeh, Javad Dargahi,
• Abstract summary: Vision-based deep learning models can deliver both tactile and visual information in a sensor-free manner. There is a lack of a comprehensive architecture capable of simultaneously segmenting the catheter from two different angles. This work proposes a novel, lightweight, multi-input, multi-output encoder-decoder-based architecture.
• Score: 0.0
• License:
• Abstract: The success rate of catheterization procedures is closely linked to the sensory data provided to the surgeon. Vision-based deep learning models can deliver both tactile and visual information in a sensor-free manner, while also being cost-effective to produce. Given the complexity of these models for devices with limited computational resources, research has focused on force estimation and catheter segmentation separately. However, there is a lack of a comprehensive architecture capable of simultaneously segmenting the catheter from two different angles and estimating the applied forces in 3D. To bridge this gap, this work proposes a novel, lightweight, multi-input, multi-output encoder-decoder-based architecture. It is designed to segment the catheter from two points of view and concurrently measure the applied forces in the x, y, and z directions. This network processes two simultaneous X-Ray images, intended to be fed by a biplane fluoroscopy system, showing a catheter's deflection from different angles. It uses two parallel sub-networks with shared parameters to output two segmentation maps corresponding to the inputs. Additionally, it leverages stereo vision to estimate the applied forces at the catheter's tip in 3D. The architecture features two input channels, two classification heads for segmentation, and a regression head for force estimation through a single end-to-end architecture. The output of all heads was assessed and compared with the literature, demonstrating state-of-the-art performance in both segmentation and force estimation. To the best of the authors' knowledge, this is the first time such a model has been proposed

Related papers

• REMOTE: Real-time Ego-motion Tracking for Various Endoscopes via Multimodal Visual Feature Learning [0.7499722271664147]
  A novel framework is proposed to perform real-time ego-motion tracking for endoscope. A multi-modal visual feature learning network is proposed to perform relative pose prediction. The absolute pose of endoscope is calculated based on relative poses.
  arXiv  Detail & Related papers  (2025-01-30T03:58:41Z)
• Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Sparse Annotations [32.69359482975795]
  This work presents a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) to obtain continuous 3D retinal layer surfaces from OCT volumes. Experiments on a synthetic dataset and three public clinical datasets show that our framework can effectively align the B-scans for potential motion correction.
  arXiv  Detail & Related papers  (2023-12-04T08:32:31Z)
• Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation [54.23510028456082]
  We propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation. The framework includes the triple-view encoder and the dual-frequency decoder.
  arXiv  Detail & Related papers  (2023-09-22T01:02:21Z)
• Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation [4.5206601127476445]
  We propose a novel convolutional neural network (CNN) and self-supervised learning (SSL) method for label-efficient 3D-to-2D segmentation. Results on different datasets demonstrate that the proposed CNN significantly improves the state of the art in scenarios with limited labeled data by up to 8% in Dice score.
  arXiv  Detail & Related papers  (2023-07-06T14:16:25Z)
• UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation [93.88170217725805]
  We propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features. Our evaluations on five benchmarks, Synapse, BTCV, ACDC, BRaTs, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy.
  arXiv  Detail & Related papers  (2022-12-08T18:59:57Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• PAENet: A Progressive Attention-Enhanced Network for 3D to 2D Retinal Vessel Segmentation [0.0]
  3D to 2D retinal vessel segmentation is a challenging problem in Optical Coherence Tomography Angiography ( OCTA) images. We propose a Progressive Attention-Enhanced Network (PAENet) based on attention mechanisms to extract rich feature representation. Our proposed algorithm achieves state-of-the-art performance compared with previous methods.
  arXiv  Detail & Related papers  (2021-08-26T10:27:25Z)
• Crosslink-Net: Double-branch Encoder Segmentation Network via Fusing Vertical and Horizontal Convolutions [58.71117402626524]
  We present a novel double-branch encoder architecture for medical image segmentation. Our architecture is inspired by two observations: 1) Since the discrimination of features learned via square convolutional kernels needs to be further improved, we propose to utilize non-square vertical and horizontal convolutional kernels. The experiments validate the effectiveness of our model on four datasets.
  arXiv  Detail & Related papers  (2021-07-24T02:58:32Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)
• Weakly Supervised 3D Classification of Chest CT using Aggregated Multi-Resolution Deep Segmentation Features [5.938730586521215]
  Weakly supervised disease classification of CT imaging suffers from poor localization owing to case-level annotations. We propose a medical classifier that leverages semantic structural concepts learned via multi-resolution segmentation feature maps.
  arXiv  Detail & Related papers  (2020-10-31T00:16:53Z)
• Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid Constrained Semi-Supervised Learning and Dual-UNet [74.22397862400177]
  We propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation. With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data.
  arXiv  Detail & Related papers  (2020-06-25T21:10:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.