Image Retrieval with Intra-Sweep Representation Learning for Neck Ultrasound Scanning Guidance

• URL: http://arxiv.org/abs/2412.07741v1
• Date: Tue, 10 Dec 2024 18:39:33 GMT
• Title: Image Retrieval with Intra-Sweep Representation Learning for Neck Ultrasound Scanning Guidance
• Authors: Wanwen Chen, Adam Schmidt, Eitan Prisman, Septimiu E. Salcudean,
• Abstract summary: Intraoperative ultrasound (US) can enhance real-time visualization in transoral robotic surgery. We propose a self-supervised contrastive learning approach to match intraoperative US views to a preoperative image database. Our method achieves 92.30% retrieval accuracy on simulated data and outperforms state-of-the-art temporal-based contrastive learning approaches.
• Score: 4.987315310656657
• License:
• Abstract: Purpose: Intraoperative ultrasound (US) can enhance real-time visualization in transoral robotic surgery. The surgeon creates a mental map with a pre-operative scan. Then, a surgical assistant performs freehand US scanning during the surgery while the surgeon operates at the remote surgical console. Communicating the target scanning plane in the surgeon's mental map is difficult. Automatic image retrieval can help match intraoperative images to preoperative scans, guiding the assistant to adjust the US probe toward the target plane. Methods: We propose a self-supervised contrastive learning approach to match intraoperative US views to a preoperative image database. We introduce a novel contrastive learning strategy that leverages intra-sweep similarity and US probe location to improve feature encoding. Additionally, our model incorporates a flexible threshold to reject unsatisfactory matches. Results: Our method achieves 92.30% retrieval accuracy on simulated data and outperforms state-of-the-art temporal-based contrastive learning approaches. Our ablation study demonstrates that using probe location in the optimization goal improves image representation, suggesting that semantic information can be extracted from probe location. We also present our approach on real patient data to show the feasibility of the proposed US probe localization system despite tissue deformation from tongue retraction. Conclusion: Our contrastive learning method, which utilizes intra-sweep similarity and US probe location, enhances US image representation learning. We also demonstrate the feasibility of using our image retrieval method to provide neck US localization on real patient US after tongue retraction.

Related papers

• Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
  We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
  arXiv  Detail & Related papers  (2024-09-18T13:40:59Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation [3.2039076408339353]
  We present an intuitive approach for generating synthetic laparoscopic images from short text prompts using diffusion-based generative models. Results on fidelity and diversity demonstrate that diffusion-based models can acquire knowledge about the style and semantics in the field of image-guided surgery.
  arXiv  Detail & Related papers  (2023-12-05T16:20:22Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• Identifying Visible Tissue in Intraoperative Ultrasound Images during Brain Surgery: A Method and Application [1.4408275800058263]
  Intraoperative ultrasound scanning is a demanding visuotactile task. It requires operators to simultaneously localise the ultrasound perspective and manually perform slight adjustments to the pose of the probe. We propose a method for the identification of the visible tissue, which enables the analysis of ultrasound probe and tissue contact.
  arXiv  Detail & Related papers  (2023-06-01T23:06:14Z)
• Live image-based neurosurgical guidance and roadmap generation using unsupervised embedding [53.992124594124896]
  We present a method for live image-only guidance leveraging a large data set of annotated neurosurgical videos. A generated roadmap encodes the common anatomical paths taken in surgeries in the training set. We trained and evaluated the proposed method with a data set of 166 transsphenoidal adenomectomy procedures.
  arXiv  Detail & Related papers  (2023-03-31T12:52:24Z)
• Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework [35.17207004351791]
  We propose a novel dual-agent framework that integrates a reinforcement learning agent and a deep learning agent. Our method can effectively interpret the US images and navigate the probe to acquire multiple standard views of the spine.
  arXiv  Detail & Related papers  (2021-11-03T12:11:27Z)
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks [48.732863591145964]
  We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
  arXiv  Detail & Related papers  (2021-10-12T21:22:24Z)
• Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning [28.17246919349759]
  We propose a framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine.
  arXiv  Detail & Related papers  (2021-03-01T03:09:17Z)
• Screen Tracking for Clinical Translation of Live Ultrasound Image Analysis Methods [2.5805793749729857]
  The proposed method captures the US image by tracking the screen with a camera fixed at the sonographer's view point and reformats the captured image to the right aspect ratio. It is hypothesized that this would enable to input such retrieved image into an image processing pipeline to extract information that can help improve the examination. This information could eventually be projected back to the sonographer's field of view in real time using, for example, an augmented reality (AR) headset.
  arXiv  Detail & Related papers  (2020-07-13T09:53:20Z)
• Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks [54.00217496410142]
  We propose an unpaired image-to-image translation where the goal is to learn the mapping between an input endoscopic image and a corresponding annotation. Our approach allows to train image segmentation models without the need to acquire expensive annotations. We test our proposed method on Endovis 2017 challenge dataset and show that it is competitive with supervised segmentation methods.
  arXiv  Detail & Related papers  (2020-07-09T01:39:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.