Hand Gesture Classification Based on Forearm Ultrasound Video Snippets Using 3D Convolutional Neural Networks

• URL: http://arxiv.org/abs/2409.16431v1
• Date: Tue, 24 Sep 2024 19:51:41 GMT
• Title: Hand Gesture Classification Based on Forearm Ultrasound Video Snippets Using 3D Convolutional Neural Networks
• Authors: Keshav Bimbraw, Ankit Talele, Haichong K. Zhang,
• Abstract summary: Forearm ultrasound offers detailed information about muscle morphology changes during hand movement which can be used to estimate hand gestures. Previous work has focused on analyzing 2-Dimensional (2D) ultrasound image frames using techniques such as convolutional neural networks (CNNs) This study uses 3D CNN based techniques to capture temporal patterns within ultrasound video segments for gesture recognition.
• Score: 2.1301560294088318
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ultrasound based hand movement estimation is a crucial area of research with applications in human-machine interaction. Forearm ultrasound offers detailed information about muscle morphology changes during hand movement which can be used to estimate hand gestures. Previous work has focused on analyzing 2-Dimensional (2D) ultrasound image frames using techniques such as convolutional neural networks (CNNs). However, such 2D techniques do not capture temporal features from segments of ultrasound data corresponding to continuous hand movements. This study uses 3D CNN based techniques to capture spatio-temporal patterns within ultrasound video segments for gesture recognition. We compared the performance of a 2D convolution-based network with (2+1)D convolution-based, 3D convolution-based, and our proposed network. Our methodology enhanced the gesture classification accuracy to 98.8 +/- 0.9%, from 96.5 +/- 2.3% compared to a network trained with 2D convolution layers. These results demonstrate the advantages of using ultrasound video snippets for improving hand gesture classification performance.

Related papers

• SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation [53.83313235792596]
  We present a new methodology for real-time semantic mapping from RGB-D sequences. It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
  arXiv  Detail & Related papers  (2023-06-28T22:36:44Z)
• Adaptive 3D Localization of 2D Freehand Ultrasound Brain Images [18.997300579859978]
  We propose AdLocUI, a framework that Adaptively Localizes 2D Ultrasound Images in the 3D anatomical atlas. We first train a convolutional neural network with 2D slices sampled from co-aligned 3D ultrasound volumes to predict their locations. We fine-tune it with 2D freehand ultrasound images using a novel unsupervised cycle consistency.
  arXiv  Detail & Related papers  (2022-09-12T17:59:41Z)
• Super Images -- A New 2D Perspective on 3D Medical Imaging Analysis [0.0]
  We present a simple yet effective 2D method to handle 3D data while efficiently embedding the 3D knowledge during training. Our method generates a super-resolution image by stitching slices side by side in the 3D image. While attaining equal, if not superior, results to 3D networks utilizing only 2D counterparts, the model complexity is reduced by around threefold.
  arXiv  Detail & Related papers  (2022-05-05T09:59:03Z)
• Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images [33.99874168018807]
  In this study, a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) is proposed to obtain continuous 3D retinal layer surfaces from OCT. Our framework achieves superior results to state-of-the-art 2D methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity.
  arXiv  Detail & Related papers  (2022-03-04T15:55:09Z)
• HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks [71.09275975580009]
  HandVoxNet++ is a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner. HandVoxNet++ relies on two hand shape representations. The first one is the 3D voxelized grid of hand shape, which does not preserve the mesh topology. We combine the advantages of both representations by aligning the hand surface to the voxelized hand shape either with a new neural Graph-Convolutions-based Mesh Registration (GCN-MeshReg) or classical segment-wise Non-Rigid Gravitational Approach (NRGA++) which
  arXiv  Detail & Related papers  (2021-07-02T17:59:54Z)
• 2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition [84.697097472401]
  We introduce Ada3D, a conditional computation framework that learns instance-specific 3D usage policies to determine frames and convolution layers to be used in a 3D network. We demonstrate that our method achieves similar accuracies to state-of-the-art 3D models while requiring 20%-50% less computation across different datasets.
  arXiv  Detail & Related papers  (2020-12-29T21:40:38Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Two-stream Fusion Model for Dynamic Hand Gesture Recognition using 3D-CNN and 2D-CNN Optical Flow guided Motion Template [0.0]
  proper detection and tracking of the moving hand become challenging due to the varied shape and size of the hand. This work basically proposes a two-stream fusion model for hand gesture recognition and a compact yet efficient motion template based on optical flow.
  arXiv  Detail & Related papers  (2020-07-17T09:20:20Z)
• Temporal Distinct Representation Learning for Action Recognition [139.93983070642412]
  Two-Dimensional Convolutional Neural Network (2D CNN) is used to characterize videos. Different frames of a video share the same 2D CNN kernels, which may result in repeated and redundant information utilization. We propose a sequential channel filtering mechanism to excite the discriminative channels of features from different frames step by step, and thus avoid repeated information extraction. Our method is evaluated on benchmark temporal reasoning datasets Something-Something V1 and V2, and it achieves visible improvements over the best competitor by 2.4% and 1.3%, respectively.
  arXiv  Detail & Related papers  (2020-07-15T11:30:40Z)
• 2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data [54.223614679807994]
  3D convolutional neural networks (CNNs) have started to show superior performance to 2D CNNs in numerous deep learning tasks. Applying transfer learning on 3D CNN is challenging due to a lack of publicly available pre-trained 3D models. In this work, we proposed a novel 2D strategical representation of volumetric data, namely 2.75D. As a result, 2D CNN networks can also be used to learn volumetric information.
  arXiv  Detail & Related papers  (2020-02-11T08:24:19Z)
• Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition [23.054444026402738]
  We present a multimodal gesture recognition method based on 3D densely convolutional networks (3D-DenseNets) and improved temporal convolutional networks (TCNs) In spatial analysis, we adopt 3D-DenseNets to learn short-term-temporal features effectively. In temporal analysis, we use TCNs to extract temporal features and employ improved Squeeze-and-Excitation Networks (SENets) to strengthen the representational power of temporal features from each TCNs' layers.
  arXiv  Detail & Related papers  (2019-12-31T23:30:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.