Point detection through multi-instance deep heatmap regression for sutures in endoscopy

• URL: http://arxiv.org/abs/2111.08468v1
• Date: Tue, 16 Nov 2021 13:45:23 GMT
• Title: Point detection through multi-instance deep heatmap regression for sutures in endoscopy
• Authors: Lalith Sharan, Gabriele Romano, Julian Brand, Halvar Kelm, Matthias Karck, Raffaele De Simone, Sandy Engelhardt
• Abstract summary: We formulate the suture detection task as a multi-instance deep heatmap regression problem. We introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains.
• Score: 0.8937790536664091
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean F1 of +0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean F1 of +0.0865 over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/. DOI:10.1007/s11548-021-02523-w. The link to the open access article can be found here: https://link.springer.com/article/10.1007%2Fs11548-021-02523-w

Related papers

• RetinaRegNet: A Zero-Shot Approach for Retinal Image Registration [10.430563602981705]
  RetinaRegNet is a zero-shot registration model designed to register retinal images with minimal overlap, large deformations, and varying image quality. We implement a two-stage registration framework to handle large deformations. Our model consistently outperformed state-of-the-art methods across all datasets.
  arXiv  Detail & Related papers  (2024-04-24T17:50:37Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• Cross-Dataset Adaptation for Instrument Classification in Cataract Surgery Videos [54.1843419649895]
  State-of-the-art models, which perform this task well on a particular dataset, perform poorly when tested on another dataset. We propose a novel end-to-end Unsupervised Domain Adaptation (UDA) method called the Barlow Adaptor. In addition, we introduce a novel loss called the Barlow Feature Alignment Loss (BFAL) which aligns features across different domains.
  arXiv  Detail & Related papers  (2023-07-31T18:14:18Z)
• Adaptation to CT Reconstruction Kernels by Enforcing Cross-domain Feature Maps Consistency [0.06117371161379209]
  We show a decrease in the COVID-19 segmentation quality of the model trained on the smooth and tested on the sharp reconstruction kernels. We propose the unsupervised adaptation method, called F-Consistency, that outperforms the previous approaches.
  arXiv  Detail & Related papers  (2022-03-28T10:00:03Z)
• Improved Heatmap-based Landmark Detection [9.0766494606964]
  This paper proposes a landmark detection network for detecting sutures in endoscopic pictures. It solves the problem of a variable number of suture points in the images. This paper performed the tests using a simulated dataset of 2708 photos and a real dataset of 2376 images.
  arXiv  Detail & Related papers  (2021-10-12T01:41:01Z)
• Automatic Polyp Segmentation via Multi-scale Subtraction Network [100.94922587360871]
  In clinical practice, precise polyp segmentation provides important information in the early detection of colorectal cancer. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. We propose a multi-scale subtraction network (MSNet) to segment polyp from colonoscopy image.
  arXiv  Detail & Related papers  (2021-08-11T07:54:07Z)
• Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation [0.9322743017642274]
  The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. We show that a task defined on sparse landmark labels improves consistency of synthesis by the generator network in both domains.
  arXiv  Detail & Related papers  (2021-07-14T19:09:50Z)
• Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy [11.457020223521605]
  This paper presents an automatic method based on Convolutional Neural Networks (CNNs) The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information.
  arXiv  Detail & Related papers  (2021-04-05T16:24:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.